[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:09:37 dist-test-slave-p3t5 krb5kdc[5166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:09:37 dist-test-slave-p3t5 krb5kdc[5166](info): set up 2 sockets
Aug 09 02:09:37 dist-test-slave-p3t5 krb5kdc[5166](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 02:09:38.485462 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:09:38 dist-test-slave-p3t5 krb5kdc[5166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:09:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:09:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:09:38.625092 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:36311
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:36311
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:38.722697 5179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:38.722990 5179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:38.723037 5179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:38.724438 5179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:09:38.724516 5179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:38.724540 5179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:38.724561 5179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:09:38.724591 5179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:09:38.726864 5179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:36311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:36311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:38.727382 5179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:38.727694 5179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:38.735155 5185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:38.735113 5184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:38.735219 5189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:38.736204 5179 server_base.cc:1034] running on GCE node
I20260809 02:09:38.736577 5179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:38.736935 5179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:38.738090 5179 hybrid_clock.cc:648] HybridClock initialized: now 1786241378738060 us; error 78 us; skew 500 ppm
Aug 09 02:09:38 dist-test-slave-p3t5 krb5kdc[5166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241378, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:38.740893 5179 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:09:38.741516 5179 webserver.cc:460] Webserver started at http://127.5.9.190:34245/ using document root <none> and password file <none>
I20260809 02:09:38.741863 5179 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:38.741919 5179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:38.742044 5179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:38.743350 5179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "c5c4941cb110447e98296ea942658c00"
format_stamp: "Formatted at 2026-08-09 02:09:38 on dist-test-slave-p3t5"
server_key: "a2613620ae7e0dc912e5d2b2b25aec9d"
server_key_iv: "5820c9affe5149c1e5a13c16eaadee3f"
server_key_version: "encryptionkey@0"
I20260809 02:09:38.743742 5179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "c5c4941cb110447e98296ea942658c00"
format_stamp: "Formatted at 2026-08-09 02:09:38 on dist-test-slave-p3t5"
server_key: "a2613620ae7e0dc912e5d2b2b25aec9d"
server_key_iv: "5820c9affe5149c1e5a13c16eaadee3f"
server_key_version: "encryptionkey@0"
I20260809 02:09:38.745894 5179 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 02:09:38.746943 5196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:38.747299 5179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:38.747421 5179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "c5c4941cb110447e98296ea942658c00"
format_stamp: "Formatted at 2026-08-09 02:09:38 on dist-test-slave-p3t5"
server_key: "a2613620ae7e0dc912e5d2b2b25aec9d"
server_key_iv: "5820c9affe5149c1e5a13c16eaadee3f"
server_key_version: "encryptionkey@0"
I20260809 02:09:38.747510 5179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:38.765117 5179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:38.767637 5179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:38.767839 5179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:38.774458 5179 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:36311
I20260809 02:09:38.774605 5250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:36311 every 8 connection(s)
I20260809 02:09:38.775108 5179 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:09:38.776713 5251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:38.779989 5251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00: Bootstrap starting.
I20260809 02:09:38.780946 5251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:38.781345 5251 log.cc:824] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:38.782112 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 5179
I20260809 02:09:38.782202 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:09:38.782459 5251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00: No bootstrap required, opened a new log
I20260809 02:09:38.782871 5158 external_mini_cluster.cc:1428] Setting key 884b1c0a845427e338cff8989870c6b7
I20260809 02:09:38.784564 5251 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c4941cb110447e98296ea942658c00" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36311 } }
I20260809 02:09:38.784847 5251 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:38.784962 5251 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c4941cb110447e98296ea942658c00, State: Initialized, Role: FOLLOWER
I20260809 02:09:38.785133 5251 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c4941cb110447e98296ea942658c00" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36311 } }
I20260809 02:09:38.785281 5251 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:09:38.785308 5251 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:09:38.785396 5251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:38.786501 5251 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c4941cb110447e98296ea942658c00" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36311 } }
I20260809 02:09:38.786652 5251 leader_election.cc:302] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5c4941cb110447e98296ea942658c00; no voters:
I20260809 02:09:38.786909 5251 leader_election.cc:288] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:09:38.786923 5256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:38.787143 5256 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [term 1 LEADER]: Becoming Leader. State: Replica: c5c4941cb110447e98296ea942658c00, State: Running, Role: LEADER
I20260809 02:09:38.787282 5256 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c4941cb110447e98296ea942658c00" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36311 } }
I20260809 02:09:38.787379 5251 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:09:38.787915 5257 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5c4941cb110447e98296ea942658c00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c4941cb110447e98296ea942658c00" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36311 } } }
I20260809 02:09:38.787985 5258 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5c4941cb110447e98296ea942658c00. Latest consensus state: current_term: 1 leader_uuid: "c5c4941cb110447e98296ea942658c00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c4941cb110447e98296ea942658c00" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36311 } } }
I20260809 02:09:38.788100 5258 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [sys.catalog]: This master's current role is: LEADER
I20260809 02:09:38.788009 5257 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00 [sys.catalog]: This master's current role is: LEADER
I20260809 02:09:38.789481 5266 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:09:38.790623 5266 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:09:38.792670 5266 catalog_manager.cc:1347] Generated new cluster ID: 3e6fb3551ae04766ab9479b254c2a782
I20260809 02:09:38.792726 5266 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:09:38 dist-test-slave-p3t5 krb5kdc[5166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:38.805143 5255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:38.784191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46136 (local address 127.5.9.190:36311)
0809 02:09:38.784676 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:38.784686 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:38.784906 (+ 220us) server_negotiation.cc:407] Connection header received
0809 02:09:38.787048 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:38.787056 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:38.787150 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:38.789134 (+ 1984us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:38.790470 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:38.792337 (+ 1867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:38.793995 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:38.794656 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:38.794775 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:38.799628 (+ 4853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:38.799634 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:38.799635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:38.799649 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:38.802077 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:38.802740 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:38.802743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:38.802744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:38.802886 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:38.803304 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:38.803305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:38.803306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:38.803653 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:38.803803 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:38.804347 (+ 544us) server_negotiation.cc:299] Negotiation successful
0809 02:09:38.804561 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":261,"threads_started":1}
I20260809 02:09:38.810608 5266 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:09:38.811343 5266 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:09:38.814651 5266 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5c4941cb110447e98296ea942658c00: Generated new TSK 0
I20260809 02:09:38.814924 5266 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:09:38.849414 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:36311
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:38.964385 5278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:38.964635 5278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:38.964665 5278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:38.966140 5278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:38.966197 5278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:38.966233 5278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:09:38.968106 5278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:36311
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:38.968470 5278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:38.968756 5278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:38.975118 5283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:38.975178 5285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:38.975118 5287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:38.975137 5278 server_base.cc:1034] running on GCE node
I20260809 02:09:38.975556 5278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:38.975841 5278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:38.977020 5278 hybrid_clock.cc:648] HybridClock initialized: now 1786241378977009 us; error 34 us; skew 500 ppm
Aug 09 02:09:38 dist-test-slave-p3t5 krb5kdc[5166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241378, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:38.980269 5278 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:09:38.980801 5278 webserver.cc:460] Webserver started at http://127.5.9.129:46761/ using document root <none> and password file <none>
I20260809 02:09:38.981053 5278 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:38.981127 5278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:38.981287 5278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:38.982640 5278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "24b357383d4c465d863fc0d469bd2d62"
format_stamp: "Formatted at 2026-08-09 02:09:38 on dist-test-slave-p3t5"
server_key: "905c83064116a0626c084238aeff10c6"
server_key_iv: "c57a80e0900a3b8db4e8d495777de6f4"
server_key_version: "encryptionkey@0"
I20260809 02:09:38.983116 5278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "24b357383d4c465d863fc0d469bd2d62"
format_stamp: "Formatted at 2026-08-09 02:09:38 on dist-test-slave-p3t5"
server_key: "905c83064116a0626c084238aeff10c6"
server_key_iv: "c57a80e0900a3b8db4e8d495777de6f4"
server_key_version: "encryptionkey@0"
I20260809 02:09:38.984855 5278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:09:38.985865 5294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:38.986117 5278 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:09:38.986238 5278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "24b357383d4c465d863fc0d469bd2d62"
format_stamp: "Formatted at 2026-08-09 02:09:38 on dist-test-slave-p3t5"
server_key: "905c83064116a0626c084238aeff10c6"
server_key_iv: "c57a80e0900a3b8db4e8d495777de6f4"
server_key_version: "encryptionkey@0"
I20260809 02:09:38.986351 5278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:39.008420 5278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:39.011036 5278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:39.011257 5278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:39.011583 5278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:09:39.012033 5278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:39.012144 5278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:39.012185 5278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:39.012214 5278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:39.027252 5278 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:38771
I20260809 02:09:39.027330 5414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:38771 every 8 connection(s)
I20260809 02:09:39.027782 5278 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:09:39 dist-test-slave-p3t5 krb5kdc[5166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241378, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:39.037395 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 5278
I20260809 02:09:39.037515 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:09:39.037716 5158 external_mini_cluster.cc:1428] Setting key ba76a92c6b3c8a484622681284d53aec
I20260809 02:09:39.039647 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:39.028914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36311 (local address 127.5.9.129:43519)
0809 02:09:39.029200 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:39.029259 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:39.030168 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:39.030455 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:39.030459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:39.031382 (+ 923us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:39.031723 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:39.031730 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:39.032449 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:39.032450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:39.033481 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:39.033484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:39.033913 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:39.033913 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:39.034055 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:39.034575 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:39.034584 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:39.036177 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:39.038042 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:39.038052 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:39.038055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:39.038316 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:39.038577 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:39.038578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:39.038579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:39.038714 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:39.039045 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:39.039054 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:39.039200 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 02:09:39.039309 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 02:09:39.039460 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":103,"threads_started":1}
I20260809 02:09:39.039649 5255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:39.028912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:43519 (local address 127.5.9.190:36311)
0809 02:09:39.028994 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:39.028997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:39.029353 (+ 356us) server_negotiation.cc:407] Connection header received
0809 02:09:39.030275 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:39.030277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:39.030289 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:39.030361 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:39.031838 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:39.032356 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:39.033569 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:39.033836 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:39.033892 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:39.036292 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:39.036294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:39.036294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:39.036314 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:39.037948 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:39.038405 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:39.038406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:39.038406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:39.038477 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:39.038791 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:39.038792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:39.038792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:39.038950 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:39.039024 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:39.039359 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 02:09:39.039501 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 02:09:39.040648 5415 heartbeater.cc:347] Connected to a master server at 127.5.9.190:36311
I20260809 02:09:39.040791 5415 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:39.040995 5415 heartbeater.cc:511] Master 127.5.9.190:36311 requested a full tablet report, sending...
I20260809 02:09:39.041903 5213 ts_manager.cc:194] Registered new tserver with Master: 24b357383d4c465d863fc0d469bd2d62 (127.5.9.129:38771)
I20260809 02:09:39.042678 5213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:43519
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:09:39.068390 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:36311
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:39.156629 5420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:39.156880 5420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:39.156909 5420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:39.158242 5420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:39.158304 5420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:39.158344 5420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:09:39.160084 5420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:36311
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:39.160439 5420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:39.160728 5420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:09:39.166424 5420 server_base.cc:1034] running on GCE node
W20260809 02:09:39.166390 5428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:39.166391 5425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:39.166414 5426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:39.166946 5420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:39.167255 5420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:39.168453 5420 hybrid_clock.cc:648] HybridClock initialized: now 1786241379168448 us; error 53 us; skew 500 ppm
Aug 09 02:09:39 dist-test-slave-p3t5 krb5kdc[5166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241379, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:39.171480 5420 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:09:39.172000 5420 webserver.cc:460] Webserver started at http://127.5.9.130:36145/ using document root <none> and password file <none>
I20260809 02:09:39.172276 5420 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:39.172348 5420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:39.172503 5420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:39.173735 5420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "7223e030350d41a6b08418d7a6b2cee0"
format_stamp: "Formatted at 2026-08-09 02:09:39 on dist-test-slave-p3t5"
server_key: "9df097d98024a7a3f936dbb5d201bdf3"
server_key_iv: "4a825ec73e0ef9a5aa0a2cef973c33e9"
server_key_version: "encryptionkey@0"
I20260809 02:09:39.174160 5420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "7223e030350d41a6b08418d7a6b2cee0"
format_stamp: "Formatted at 2026-08-09 02:09:39 on dist-test-slave-p3t5"
server_key: "9df097d98024a7a3f936dbb5d201bdf3"
server_key_iv: "4a825ec73e0ef9a5aa0a2cef973c33e9"
server_key_version: "encryptionkey@0"
I20260809 02:09:39.175674 5420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:09:39.176505 5435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:39.176683 5420 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:09:39.176798 5420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "7223e030350d41a6b08418d7a6b2cee0"
format_stamp: "Formatted at 2026-08-09 02:09:39 on dist-test-slave-p3t5"
server_key: "9df097d98024a7a3f936dbb5d201bdf3"
server_key_iv: "4a825ec73e0ef9a5aa0a2cef973c33e9"
server_key_version: "encryptionkey@0"
I20260809 02:09:39.176882 5420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:39.190450 5420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:39.193087 5420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:39.193271 5420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:39.193544 5420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:09:39.193969 5420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:39.194031 5420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:39.194077 5420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:39.194103 5420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:39.201933 5420 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:35285
I20260809 02:09:39.201984 5557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:35285 every 8 connection(s)
I20260809 02:09:39.202478 5420 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:09:39.204977 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 5420
I20260809 02:09:39.205066 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:09:39.205401 5158 external_mini_cluster.cc:1428] Setting key b7dabdf3aa0e8d89d31cf19ff82b97d9
Aug 09 02:09:39 dist-test-slave-p3t5 krb5kdc[5166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241379, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:39.214417 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:39.203355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36311 (local address 127.5.9.130:54547)
0809 02:09:39.203537 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:39.203583 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:39.204104 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:39.204372 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:39.204375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:39.205002 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:39.205296 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:39.205301 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:39.206318 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:39.206319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:39.207193 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:39.207196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:39.207830 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:39.207832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:39.208017 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:39.208666 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:39.208676 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:39.210515 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:39.212725 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:39.212733 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:39.212736 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:39.213009 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:39.213256 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:39.213257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:39.213258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:39.213384 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:39.213791 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:39.213806 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:39.213953 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 02:09:39.214069 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 02:09:39.214223 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
I20260809 02:09:39.214404 5255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:39.203448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:54547 (local address 127.5.9.190:36311)
0809 02:09:39.203518 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:39.203520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:39.203717 (+ 197us) server_negotiation.cc:407] Connection header received
0809 02:09:39.204229 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:39.204230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:39.204244 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:39.204311 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:39.205407 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:39.206222 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:39.207298 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:39.207662 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:39.207737 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:39.210640 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:39.210642 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:39.210643 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:39.210653 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:39.212618 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:39.213101 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:39.213103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:39.213103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:39.213180 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:39.213458 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:39.213459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:39.213459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:39.213709 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:39.213791 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:39.214125 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 02:09:39.214248 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:09:39.215086 5558 heartbeater.cc:347] Connected to a master server at 127.5.9.190:36311
I20260809 02:09:39.215194 5558 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:39.215411 5558 heartbeater.cc:511] Master 127.5.9.190:36311 requested a full tablet report, sending...
I20260809 02:09:39.215881 5213 ts_manager.cc:194] Registered new tserver with Master: 7223e030350d41a6b08418d7a6b2cee0 (127.5.9.130:35285)
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
I20260809 02:09:39.216286 5213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:54547
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:09:39.236416 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:36311
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:39.325810 5563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:39.326088 5563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:39.326133 5563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:39.327406 5563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:39.327483 5563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:39.327533 5563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:09:39.329104 5563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:36311
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:39.329401 5563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:39.329728 5563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:39.335525 5571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:39.335493 5568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:39.335600 5563 server_base.cc:1034] running on GCE node
W20260809 02:09:39.335517 5569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:39.336027 5563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:39.336254 5563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:39.337412 5563 hybrid_clock.cc:648] HybridClock initialized: now 1786241379337380 us; error 58 us; skew 500 ppm
Aug 09 02:09:39 dist-test-slave-p3t5 krb5kdc[5166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241379, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:39.340276 5563 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:09:39.340705 5563 webserver.cc:460] Webserver started at http://127.5.9.131:34643/ using document root <none> and password file <none>
I20260809 02:09:39.340902 5563 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:39.340945 5563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:39.341038 5563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:39.342173 5563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "15af39c9bb8a44cc8ef767ee2879ed8b"
format_stamp: "Formatted at 2026-08-09 02:09:39 on dist-test-slave-p3t5"
server_key: "68a0674fe42b613e4f1e8181a85077b9"
server_key_iv: "58cd2f84a573447b6882d9dd1ed5d30e"
server_key_version: "encryptionkey@0"
I20260809 02:09:39.342533 5563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "15af39c9bb8a44cc8ef767ee2879ed8b"
format_stamp: "Formatted at 2026-08-09 02:09:39 on dist-test-slave-p3t5"
server_key: "68a0674fe42b613e4f1e8181a85077b9"
server_key_iv: "58cd2f84a573447b6882d9dd1ed5d30e"
server_key_version: "encryptionkey@0"
I20260809 02:09:39.343909 5563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:09:39.344612 5579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:39.344831 5563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:09:39.344930 5563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "15af39c9bb8a44cc8ef767ee2879ed8b"
format_stamp: "Formatted at 2026-08-09 02:09:39 on dist-test-slave-p3t5"
server_key: "68a0674fe42b613e4f1e8181a85077b9"
server_key_iv: "58cd2f84a573447b6882d9dd1ed5d30e"
server_key_version: "encryptionkey@0"
I20260809 02:09:39.344992 5563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:39.353981 5563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:39.356138 5563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:39.356264 5563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:39.356494 5563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:09:39.356841 5563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:39.356878 5563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:39.356904 5563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:39.356961 5563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:39.364240 5563 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:34255
I20260809 02:09:39.364372 5702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:34255 every 8 connection(s)
I20260809 02:09:39.364742 5563 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
Aug 09 02:09:39 dist-test-slave-p3t5 krb5kdc[5166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241379, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:39.372830 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 5563
I20260809 02:09:39.372941 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:09:39.373135 5158 external_mini_cluster.cc:1428] Setting key 428a4d65ce014b146534abab827a5d93
I20260809 02:09:39.375591 5255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:39.365759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:48207 (local address 127.5.9.190:36311)
0809 02:09:39.365832 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:39.365835 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:39.366137 (+ 302us) server_negotiation.cc:407] Connection header received
0809 02:09:39.366769 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:39.366770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:39.366780 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:39.366835 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:39.367859 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:39.368365 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:39.369410 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:39.369596 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:39.369684 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:39.372256 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:39.372257 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:39.372257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:39.372265 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:39.373988 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:39.374550 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:39.374551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:39.374552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:39.374607 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:39.374884 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:39.374885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:39.374885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:39.375022 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:39.375103 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:39.375402 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 02:09:39.375490 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:09:39.375823 5705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:39.365739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36311 (local address 127.5.9.131:48207)
0809 02:09:39.365989 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:39.366031 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:39.366664 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:39.366916 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:39.366919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:39.367537 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:39.367762 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:39.367768 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:39.368459 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:39.368460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:39.369327 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:39.369329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:39.369754 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:39.369755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:39.369884 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:39.370575 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:39.370588 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:39.372138 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:39.374086 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:39.374097 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:39.374099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:39.374467 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:39.374680 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:39.374681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:39.374682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:39.374812 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:39.375095 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:39.375102 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:39.375256 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 02:09:39.375370 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 02:09:39.375564 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":111,"threads_started":1}
I20260809 02:09:39.376288 5703 heartbeater.cc:347] Connected to a master server at 127.5.9.190:36311
I20260809 02:09:39.376391 5703 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:39.376549 5703 heartbeater.cc:511] Master 127.5.9.190:36311 requested a full tablet report, sending...
I20260809 02:09:39.376947 5213 ts_manager.cc:194] Registered new tserver with Master: 15af39c9bb8a44cc8ef767ee2879ed8b (127.5.9.131:34255)
I20260809 02:09:39.377307 5213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:48207
I20260809 02:09:39.385727 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:09:39 dist-test-slave-p3t5 krb5kdc[5166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241379, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:09:39 dist-test-slave-p3t5 krb5kdc[5166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241379, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.9.129@KRBTEST.COM
I20260809 02:09:39.413537 5710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:39.404667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57586 (local address 127.5.9.129:38771)
0809 02:09:39.404978 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:39.404984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:39.404997 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:09:39.405027 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:39.405029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:39.405044 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:39.405118 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:39.405807 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:39.406674 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:39.407435 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:39.407813 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:39.407884 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:39.410641 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:39.410653 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:39.410654 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:39.410662 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:39.412289 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:39.412670 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:39.412674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:39.412674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:39.412777 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:39.412986 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:39.412986 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:39.412987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:39.413141 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:39.413259 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:39.413319 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 02:09:39.413393 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":170,"threads_started":1}
W20260809 02:09:39.414142 5323 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57586
Aug 09 02:09:39 dist-test-slave-p3t5 krb5kdc[5166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:09:39 dist-test-slave-p3t5 krb5kdc[5166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.129@KRBTEST.COM
I20260809 02:09:39.440653 5710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:39.432637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57590 (local address 127.5.9.129:38771)
0809 02:09:39.432716 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:39.432719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:39.432776 (+ 57us) server_negotiation.cc:407] Connection header received
0809 02:09:39.432857 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:39.432859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:39.432873 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:39.432949 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:39.433571 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:39.434143 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:39.434929 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:39.435147 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:39.435229 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:39.437887 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:39.437889 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:39.437889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:39.437906 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:39.439383 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:39.439785 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:39.439786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:39.439787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:39.439854 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:39.440101 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:39.440102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:39.440102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:39.440215 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:39.440351 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:39.440446 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 02:09:39.440541 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:09:39.441354 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 5278
I20260809 02:09:39.448637 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 5420
I20260809 02:09:39.455830 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 5563
I20260809 02:09:39.462036 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 5179
2026-08-09T02:09:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2070 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:09:39 dist-test-slave-p3t5 krb5kdc[5717](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:09:39 dist-test-slave-p3t5 krb5kdc[5717](info): set up 2 sockets
Aug 09 02:09:39 dist-test-slave-p3t5 krb5kdc[5717](info): commencing operation
krb5kdc: starting...
W20260809 02:09:40.533466 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:09:40 dist-test-slave-p3t5 krb5kdc[5717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:09:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:09:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:09:40.672519 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:42801
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:42801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:40.758972 5731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:40.759220 5731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:40.759245 5731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:40.760433 5731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:09:40.760474 5731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:40.760489 5731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:40.760502 5731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:09:40.760514 5731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:09:40.762076 5731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:42801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:42801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:40.762310 5731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:40.762511 5731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:40.768158 5740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:40.768247 5737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:40.768277 5731 server_base.cc:1034] running on GCE node
W20260809 02:09:40.768222 5738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:40.768647 5731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:40.769001 5731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:40.770207 5731 hybrid_clock.cc:648] HybridClock initialized: now 1786241380770193 us; error 47 us; skew 500 ppm
Aug 09 02:09:40 dist-test-slave-p3t5 krb5kdc[5717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241380, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:40.773063 5731 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:09:40.773518 5731 webserver.cc:460] Webserver started at http://127.5.9.190:46663/ using document root <none> and password file <none>
I20260809 02:09:40.773792 5731 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:40.773842 5731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:40.773953 5731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:40.775079 5731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "a6f69da8be774de5ac12ffa8266b1743"
format_stamp: "Formatted at 2026-08-09 02:09:40 on dist-test-slave-p3t5"
server_key: "3644f06277e6d292bc038685f18211b1"
server_key_iv: "4daa3961b3f87f9d14fabdfd6c3dd908"
server_key_version: "encryptionkey@0"
I20260809 02:09:40.775424 5731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "a6f69da8be774de5ac12ffa8266b1743"
format_stamp: "Formatted at 2026-08-09 02:09:40 on dist-test-slave-p3t5"
server_key: "3644f06277e6d292bc038685f18211b1"
server_key_iv: "4daa3961b3f87f9d14fabdfd6c3dd908"
server_key_version: "encryptionkey@0"
I20260809 02:09:40.776913 5731 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:09:40.777761 5747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:40.777999 5731 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:40.778122 5731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "a6f69da8be774de5ac12ffa8266b1743"
format_stamp: "Formatted at 2026-08-09 02:09:40 on dist-test-slave-p3t5"
server_key: "3644f06277e6d292bc038685f18211b1"
server_key_iv: "4daa3961b3f87f9d14fabdfd6c3dd908"
server_key_version: "encryptionkey@0"
I20260809 02:09:40.778220 5731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:40.801725 5731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:40.804021 5731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:40.804214 5731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:40.808724 5731 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:42801
I20260809 02:09:40.808826 5802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:42801 every 8 connection(s)
I20260809 02:09:40.809222 5731 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:09:40.810253 5803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:40.812994 5803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743: Bootstrap starting.
I20260809 02:09:40.813710 5803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:40.814054 5803 log.cc:824] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:40.814934 5803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743: No bootstrap required, opened a new log
I20260809 02:09:40.816679 5803 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f69da8be774de5ac12ffa8266b1743" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42801 } }
I20260809 02:09:40.816839 5803 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:40.816864 5803 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f69da8be774de5ac12ffa8266b1743, State: Initialized, Role: FOLLOWER
I20260809 02:09:40.817055 5803 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f69da8be774de5ac12ffa8266b1743" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42801 } }
I20260809 02:09:40.817143 5803 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:09:40.817168 5803 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:09:40.817245 5803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:40.817878 5803 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f69da8be774de5ac12ffa8266b1743" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42801 } }
I20260809 02:09:40.817978 5803 leader_election.cc:302] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6f69da8be774de5ac12ffa8266b1743; no voters:
I20260809 02:09:40.818189 5803 leader_election.cc:288] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:09:40.818259 5806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:40.818559 5803 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:09:40.818873 5806 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [term 1 LEADER]: Becoming Leader. State: Replica: a6f69da8be774de5ac12ffa8266b1743, State: Running, Role: LEADER
I20260809 02:09:40.818986 5806 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f69da8be774de5ac12ffa8266b1743" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42801 } }
I20260809 02:09:40.819109 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 5731
I20260809 02:09:40.819219 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:09:40.819418 5158 external_mini_cluster.cc:1428] Setting key 1c6eda485dccf8b89629acafdba83b9b
I20260809 02:09:40.819582 5808 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6f69da8be774de5ac12ffa8266b1743. Latest consensus state: current_term: 1 leader_uuid: "a6f69da8be774de5ac12ffa8266b1743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f69da8be774de5ac12ffa8266b1743" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42801 } } }
I20260809 02:09:40.819748 5807 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6f69da8be774de5ac12ffa8266b1743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f69da8be774de5ac12ffa8266b1743" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42801 } } }
I20260809 02:09:40.819832 5807 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [sys.catalog]: This master's current role is: LEADER
I20260809 02:09:40.819778 5808 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743 [sys.catalog]: This master's current role is: LEADER
I20260809 02:09:40.820307 5820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:09:40.821142 5820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:09:40.822896 5820 catalog_manager.cc:1347] Generated new cluster ID: 5e223158918a4138ac43bb8dcd2388d3
I20260809 02:09:40.822953 5820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:09:40 dist-test-slave-p3t5 krb5kdc[5717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:40.832794 5824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:40.820160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41604 (local address 127.5.9.190:42801)
0809 02:09:40.820610 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:40.820621 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:40.820664 (+ 43us) server_negotiation.cc:407] Connection header received
0809 02:09:40.820804 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:40.820812 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:40.821127 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:40.821312 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:40.822217 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:40.823636 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:40.824719 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:40.825397 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:40.825520 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:40.828707 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:40.828713 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:40.828713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:40.828727 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:40.830936 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:40.831408 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:40.831411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:40.831411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:40.831536 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:40.831767 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:40.831768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:40.831768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:40.832097 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:40.832265 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:40.832389 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 02:09:40.832536 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":74,"threads_started":1}
I20260809 02:09:40.845558 5820 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:09:40.846226 5820 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:09:40.851668 5820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6f69da8be774de5ac12ffa8266b1743: Generated new TSK 0
I20260809 02:09:40.851915 5820 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:09:40.887017 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42801
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:40.976723 5831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:40.977020 5831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:40.977069 5831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:40.978365 5831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:40.978456 5831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:40.978505 5831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:09:40.980123 5831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42801
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:40.980423 5831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:40.980684 5831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:40.986449 5839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:40.986415 5836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:40.986402 5837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:40.986644 5831 server_base.cc:1034] running on GCE node
I20260809 02:09:40.986882 5831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:40.987108 5831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:40.988287 5831 hybrid_clock.cc:648] HybridClock initialized: now 1786241380988266 us; error 34 us; skew 500 ppm
Aug 09 02:09:40 dist-test-slave-p3t5 krb5kdc[5717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241380, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:40.991214 5831 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:09:40.991655 5831 webserver.cc:460] Webserver started at http://127.5.9.129:35569/ using document root <none> and password file <none>
I20260809 02:09:40.991856 5831 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:40.991899 5831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:40.991997 5831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:40.993072 5831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "f654426af8e24d208c891c8404d751cf"
format_stamp: "Formatted at 2026-08-09 02:09:40 on dist-test-slave-p3t5"
server_key: "788a78d5eb8bdd3b05b02d839eb41565"
server_key_iv: "145c0afd41c6b79f43772e662200b016"
server_key_version: "encryptionkey@0"
I20260809 02:09:40.993492 5831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "f654426af8e24d208c891c8404d751cf"
format_stamp: "Formatted at 2026-08-09 02:09:40 on dist-test-slave-p3t5"
server_key: "788a78d5eb8bdd3b05b02d839eb41565"
server_key_iv: "145c0afd41c6b79f43772e662200b016"
server_key_version: "encryptionkey@0"
I20260809 02:09:40.994953 5831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:09:40.995695 5846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:40.995887 5831 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:09:40.996002 5831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "f654426af8e24d208c891c8404d751cf"
format_stamp: "Formatted at 2026-08-09 02:09:40 on dist-test-slave-p3t5"
server_key: "788a78d5eb8bdd3b05b02d839eb41565"
server_key_iv: "145c0afd41c6b79f43772e662200b016"
server_key_version: "encryptionkey@0"
I20260809 02:09:40.996084 5831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:41.023244 5831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:41.025550 5831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:41.025771 5831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:41.026049 5831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:09:41.026398 5831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:41.026456 5831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:41.026506 5831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:41.026530 5831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:41.033008 5831 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:38479
I20260809 02:09:41.033059 5963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:38479 every 8 connection(s)
I20260809 02:09:41.033428 5831 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:09:41.033906 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 5831
I20260809 02:09:41.034040 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:09:41.034221 5158 external_mini_cluster.cc:1428] Setting key 52a052ffc1a1f7112f9a07a9b49e3f4f
Aug 09 02:09:41 dist-test-slave-p3t5 krb5kdc[5717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241380, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
I20260809 02:09:41.046051 5824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.034391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:49947 (local address 127.5.9.190:42801)
0809 02:09:41.034450 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:41.034452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:41.034750 (+ 298us) server_negotiation.cc:407] Connection header received
0809 02:09:41.036385 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:41.036387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:41.036402 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:41.036462 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:41.037389 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.038237 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.039353 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.039700 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.039784 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.042728 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:41.042730 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:41.042731 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:41.042742 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:41.044400 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:41.044867 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:41.044869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:41.044869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:41.044937 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:41.045229 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:41.045230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:41.045230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:41.045366 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:41.045448 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:41.045833 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 02:09:41.045950 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 02:09:41.046248 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.034363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:42801 (local address 127.5.9.129:49947)
0809 02:09:41.034576 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:41.034638 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:41.036218 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:41.036553 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:41.036558 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:41.037012 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:41.037268 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:41.037274 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.038381 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.038382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:41.039249 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:41.039252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.039816 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.039818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:41.039956 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.040617 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:41.040630 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:41.042590 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:41.044503 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:41.044512 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:41.044515 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:41.044779 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:41.045018 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:41.045019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:41.045019 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:41.045150 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:41.045438 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:41.045446 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:41.045658 (+ 212us) client_negotiation.cc:769] Sending connection context
0809 02:09:41.045789 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 02:09:41.045981 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":119,"threads_started":1}
I20260809 02:09:41.046977 5964 heartbeater.cc:347] Connected to a master server at 127.5.9.190:42801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
I20260809 02:09:41.047097 5964 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:41.047319 5964 heartbeater.cc:511] Master 127.5.9.190:42801 requested a full tablet report, sending...
I20260809 02:09:41.047984 5764 ts_manager.cc:194] Registered new tserver with Master: f654426af8e24d208c891c8404d751cf (127.5.9.129:38479)
I20260809 02:09:41.048952 5764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:49947
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:09:41.064730 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42801
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:41.152136 5969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:41.152431 5969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:41.152474 5969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:41.153724 5969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:41.153806 5969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:41.153853 5969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:09:41.155462 5969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42801
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:41.155768 5969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:41.156042 5969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:41.161520 5977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:41.161540 5979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:41.161536 5976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:41.161814 5969 server_base.cc:1034] running on GCE node
I20260809 02:09:41.162045 5969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:41.162267 5969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:41.163452 5969 hybrid_clock.cc:648] HybridClock initialized: now 1786241381163435 us; error 46 us; skew 500 ppm
Aug 09 02:09:41 dist-test-slave-p3t5 krb5kdc[5717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241381, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:41.169516 5969 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:09:41.170233 5969 webserver.cc:460] Webserver started at http://127.5.9.130:40715/ using document root <none> and password file <none>
I20260809 02:09:41.170544 5969 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:41.170625 5969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:41.170799 5969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:41.172422 5969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "828bc9cc840949e792ccefb3d4909dba"
format_stamp: "Formatted at 2026-08-09 02:09:41 on dist-test-slave-p3t5"
server_key: "1d9e57c7540c69b5ba71b85c05367a38"
server_key_iv: "73dab2fd9196c0912357e50730a58e76"
server_key_version: "encryptionkey@0"
I20260809 02:09:41.172894 5969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "828bc9cc840949e792ccefb3d4909dba"
format_stamp: "Formatted at 2026-08-09 02:09:41 on dist-test-slave-p3t5"
server_key: "1d9e57c7540c69b5ba71b85c05367a38"
server_key_iv: "73dab2fd9196c0912357e50730a58e76"
server_key_version: "encryptionkey@0"
I20260809 02:09:41.174933 5969 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260809 02:09:41.176021 5986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:41.176239 5969 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:41.176338 5969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "828bc9cc840949e792ccefb3d4909dba"
format_stamp: "Formatted at 2026-08-09 02:09:41 on dist-test-slave-p3t5"
server_key: "1d9e57c7540c69b5ba71b85c05367a38"
server_key_iv: "73dab2fd9196c0912357e50730a58e76"
server_key_version: "encryptionkey@0"
I20260809 02:09:41.176424 5969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:41.210592 5969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:41.213637 5969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:41.213824 5969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:41.214126 5969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:09:41.214474 5969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:41.214514 5969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:41.214550 5969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:41.214568 5969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:41.222129 5969 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:43293
I20260809 02:09:41.222193 6103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:43293 every 8 connection(s)
I20260809 02:09:41.222554 5969 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:09:41 dist-test-slave-p3t5 krb5kdc[5717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241381, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:41.231412 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 5969
I20260809 02:09:41.231518 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:09:41.231679 5158 external_mini_cluster.cc:1428] Setting key 37b47ded7e26439f905b92762f1c5012
I20260809 02:09:41.235996 5824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.223464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:35389 (local address 127.5.9.190:42801)
0809 02:09:41.223512 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:41.223514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:41.223813 (+ 299us) server_negotiation.cc:407] Connection header received
0809 02:09:41.225201 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:41.225204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:41.225219 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:41.225301 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:41.226419 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.227152 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.228508 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.228846 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.228927 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.231865 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:41.231867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:41.231868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:41.231877 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:41.233987 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:41.234612 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:41.234614 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:41.234614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:41.234698 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:41.235044 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:41.235045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:41.235046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:41.235237 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:41.235319 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:41.235778 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 02:09:41.235895 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:09:41.236312 6106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.223445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:42801 (local address 127.5.9.130:35389)
0809 02:09:41.223622 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:41.223693 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:41.225073 (+ 1380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:41.225392 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:41.225397 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:41.225875 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:41.226183 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:41.226190 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.227261 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.227263 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:41.228418 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:41.228421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.228941 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.228943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:41.229102 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.229715 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:41.229727 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:41.231704 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:41.234097 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:41.234105 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:41.234108 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:41.234518 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:41.234784 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:41.234785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:41.234785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:41.234974 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:41.235324 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:41.235332 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:41.235539 (+ 207us) client_negotiation.cc:769] Sending connection context
0809 02:09:41.235721 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 02:09:41.235982 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260809 02:09:41.236837 6104 heartbeater.cc:347] Connected to a master server at 127.5.9.190:42801
I20260809 02:09:41.236968 6104 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:41.237176 6104 heartbeater.cc:511] Master 127.5.9.190:42801 requested a full tablet report, sending...
I20260809 02:09:41.237713 5764 ts_manager.cc:194] Registered new tserver with Master: 828bc9cc840949e792ccefb3d4909dba (127.5.9.130:43293)
I20260809 02:09:41.238292 5764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:35389
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:09:41.265167 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42801
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:41.353976 6109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:41.354225 6109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:41.354254 6109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:41.355585 6109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:41.355645 6109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:41.355677 6109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:09:41.357347 6109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42801
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:41.357697 6109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:41.357992 6109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:41.363519 6114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:41.363581 6115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:41.363831 6109 server_base.cc:1034] running on GCE node
W20260809 02:09:41.363519 6118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:41.364117 6109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:41.364351 6109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:41.365593 6109 hybrid_clock.cc:648] HybridClock initialized: now 1786241381365570 us; error 43 us; skew 500 ppm
Aug 09 02:09:41 dist-test-slave-p3t5 krb5kdc[5717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241381, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:41.368507 6109 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:09:41.369015 6109 webserver.cc:460] Webserver started at http://127.5.9.131:37569/ using document root <none> and password file <none>
I20260809 02:09:41.369277 6109 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:41.369333 6109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:41.369517 6109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:41.370715 6109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "8d9b956ecee449ee8efdae77dac88afc"
format_stamp: "Formatted at 2026-08-09 02:09:41 on dist-test-slave-p3t5"
server_key: "9df0fda038dd5dce2996e5a6ce084239"
server_key_iv: "9636704c4c407a938a47a4d20f51999e"
server_key_version: "encryptionkey@0"
I20260809 02:09:41.371131 6109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "8d9b956ecee449ee8efdae77dac88afc"
format_stamp: "Formatted at 2026-08-09 02:09:41 on dist-test-slave-p3t5"
server_key: "9df0fda038dd5dce2996e5a6ce084239"
server_key_iv: "9636704c4c407a938a47a4d20f51999e"
server_key_version: "encryptionkey@0"
I20260809 02:09:41.372695 6109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:09:41.373533 6126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:41.373755 6109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:09:41.373903 6109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "8d9b956ecee449ee8efdae77dac88afc"
format_stamp: "Formatted at 2026-08-09 02:09:41 on dist-test-slave-p3t5"
server_key: "9df0fda038dd5dce2996e5a6ce084239"
server_key_iv: "9636704c4c407a938a47a4d20f51999e"
server_key_version: "encryptionkey@0"
I20260809 02:09:41.373993 6109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:41.387149 6109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:41.389890 6109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:41.390094 6109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:41.390368 6109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:09:41.390731 6109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:41.390791 6109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:41.390829 6109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:41.390878 6109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:41.396762 6109 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:35631
I20260809 02:09:41.396854 6241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:35631 every 8 connection(s)
I20260809 02:09:41.397267 6109 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:09:41.402038 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 6109
I20260809 02:09:41.402165 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:09:41.402374 5158 external_mini_cluster.cc:1428] Setting key b7dad78a12f777e403bccf8ce4226813
Aug 09 02:09:41 dist-test-slave-p3t5 krb5kdc[5717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241381, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:41.408051 5824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.398168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:45179 (local address 127.5.9.190:42801)
0809 02:09:41.398247 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:41.398250 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:41.398525 (+ 275us) server_negotiation.cc:407] Connection header received
0809 02:09:41.399554 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:41.399555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:41.399567 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:41.399620 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:41.400315 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.400826 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.402291 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.402530 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.402619 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.405014 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:41.405016 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:41.405017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:41.405025 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:41.406490 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:41.406958 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:41.406959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:41.406960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:41.407029 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:41.407314 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:41.407315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:41.407315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:41.407454 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:41.407534 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:41.407849 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 02:09:41.407946 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:09:41.408278 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.398118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:42801 (local address 127.5.9.131:45179)
0809 02:09:41.398359 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:41.398417 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:41.399445 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:41.399692 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:41.399695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:41.399999 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:41.400205 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:41.400231 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.400946 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.400947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:41.402126 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:41.402131 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.402632 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.402633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:41.402778 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.403385 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:41.403394 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:41.404879 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:41.406583 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:41.406591 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:41.406594 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:41.406866 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:41.407101 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:41.407102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:41.407103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:41.407230 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:41.407524 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:41.407532 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:41.407695 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 02:09:41.407824 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 02:09:41.408020 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":117,"threads_started":1}
I20260809 02:09:41.408815 6242 heartbeater.cc:347] Connected to a master server at 127.5.9.190:42801
I20260809 02:09:41.408942 6242 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:41.409147 6242 heartbeater.cc:511] Master 127.5.9.190:42801 requested a full tablet report, sending...
I20260809 02:09:41.409583 5764 ts_manager.cc:194] Registered new tserver with Master: 8d9b956ecee449ee8efdae77dac88afc (127.5.9.131:35631)
I20260809 02:09:41.410025 5764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:45179
I20260809 02:09:41.414363 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:09:41.423000 5824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.416359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41614 (local address 127.5.9.190:42801)
0809 02:09:41.416532 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:41.416534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:41.416631 (+ 97us) server_negotiation.cc:407] Connection header received
0809 02:09:41.416713 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:41.416714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:41.416725 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:41.416784 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:41.417389 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.417984 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.418795 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.419002 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.419083 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.420384 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:41.420385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:41.420386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:41.420393 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:41.421720 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:41.422081 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:41.422083 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:41.422083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:41.422157 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:41.422382 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:41.422382 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:41.422383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:41.422556 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:41.422672 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:41.422794 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 02:09:41.422889 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260809 02:09:41.425783 5764 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:09:41.427625 5764 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:09:41.435238 6256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.430615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:38479 (local address 127.0.0.1:56276)
0809 02:09:41.431074 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:41.431125 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:41.431204 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:41.431431 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:41.431434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:41.431437 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:41.431528 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:41.431531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.432719 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.432720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:41.434029 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:41.434032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.434861 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.434862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:41.434965 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.434981 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 02:09:41.435084 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 02:09:41.435147 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":83,"threads_started":1}
I20260809 02:09:41.435246 6258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.430846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56276 (local address 127.5.9.129:38479)
0809 02:09:41.431169 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:41.431173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:41.431185 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:09:41.431296 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:41.431298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:41.431316 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:41.431401 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:41.431613 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.432606 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.434128 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.434771 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.434870 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.434938 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:41.435105 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 02:09:41.435149 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":112,"threads_started":1}
I20260809 02:09:41.435469 6255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.430615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:35631 (local address 127.0.0.1:32954)
0809 02:09:41.431323 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:41.431332 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:41.431384 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:41.431656 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:41.431656 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:41.431658 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:41.431713 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:41.431714 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.433082 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.433083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:41.434163 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:41.434165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.435184 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.435186 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:41.435252 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.435256 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:09:41.435306 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 02:09:41.435346 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":2,"spinlock_wait_cycles":768,"thread_start_us":100,"threads_started":1}
I20260809 02:09:41.435602 6259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.430915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58048 (local address 127.5.9.130:43293)
0809 02:09:41.431068 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:41.431073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:41.431814 (+ 741us) server_negotiation.cc:407] Connection header received
0809 02:09:41.431934 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:41.431937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:41.431953 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:41.432039 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:41.432171 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.433169 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.434428 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.435160 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.435228 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.435280 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:41.435458 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 02:09:41.435509 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260809 02:09:41.435811 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.430630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:43293 (local address 127.0.0.1:58048)
0809 02:09:41.431483 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:41.431492 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:41.431847 (+ 355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:41.432062 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:41.432063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:41.432065 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:41.432109 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:41.432110 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.433280 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.433281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:41.434324 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:41.434327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.435314 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.435314 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:41.435365 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.435368 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:09:41.435697 (+ 329us) client_negotiation.cc:240] Negotiation successful
0809 02:09:41.435746 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"mutex_wait_us":1,"thread_start_us":117,"threads_started":1}
I20260809 02:09:41.435861 6257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.430825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32954 (local address 127.5.9.131:35631)
0809 02:09:41.431055 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:41.431060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:41.431401 (+ 341us) server_negotiation.cc:407] Connection header received
0809 02:09:41.431548 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:41.431550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:41.431568 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:41.431658 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:41.431772 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.432946 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.434251 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.435003 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.435083 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.435143 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:41.435732 (+ 589us) server_negotiation.cc:299] Negotiation successful
0809 02:09:41.435777 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":112,"threads_started":1}
I20260809 02:09:41.436542 5895 tablet_service.cc:1467] Processing CreateTablet for tablet 90b03aa578034bbb8e72143c45ac0b43 (DEFAULT_TABLE table=test-table [id=772bcbd8de464849b8cc71b29055c016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:41.436868 5895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90b03aa578034bbb8e72143c45ac0b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:41.438055 6035 tablet_service.cc:1467] Processing CreateTablet for tablet 90b03aa578034bbb8e72143c45ac0b43 (DEFAULT_TABLE table=test-table [id=772bcbd8de464849b8cc71b29055c016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:41.438338 6035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90b03aa578034bbb8e72143c45ac0b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:41.439266 6174 tablet_service.cc:1467] Processing CreateTablet for tablet 90b03aa578034bbb8e72143c45ac0b43 (DEFAULT_TABLE table=test-table [id=772bcbd8de464849b8cc71b29055c016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:41.439630 6174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90b03aa578034bbb8e72143c45ac0b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:41.439906 6260 tablet_bootstrap.cc:492] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf: Bootstrap starting.
I20260809 02:09:41.440565 6260 tablet_bootstrap.cc:654] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:41.441098 6260 log.cc:824] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:41.441748 6262 tablet_bootstrap.cc:492] T 90b03aa578034bbb8e72143c45ac0b43 P 828bc9cc840949e792ccefb3d4909dba: Bootstrap starting.
I20260809 02:09:41.442046 6260 tablet_bootstrap.cc:492] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf: No bootstrap required, opened a new log
I20260809 02:09:41.442147 6260 ts_tablet_manager.cc:1397] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:09:41.442924 6262 tablet_bootstrap.cc:654] T 90b03aa578034bbb8e72143c45ac0b43 P 828bc9cc840949e792ccefb3d4909dba: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:41.443516 6262 log.cc:824] T 90b03aa578034bbb8e72143c45ac0b43 P 828bc9cc840949e792ccefb3d4909dba: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:41.443813 6263 tablet_bootstrap.cc:492] T 90b03aa578034bbb8e72143c45ac0b43 P 8d9b956ecee449ee8efdae77dac88afc: Bootstrap starting.
I20260809 02:09:41.444438 6262 tablet_bootstrap.cc:492] T 90b03aa578034bbb8e72143c45ac0b43 P 828bc9cc840949e792ccefb3d4909dba: No bootstrap required, opened a new log
I20260809 02:09:41.444535 6262 ts_tablet_manager.cc:1397] T 90b03aa578034bbb8e72143c45ac0b43 P 828bc9cc840949e792ccefb3d4909dba: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 02:09:41.444773 6263 tablet_bootstrap.cc:654] T 90b03aa578034bbb8e72143c45ac0b43 P 8d9b956ecee449ee8efdae77dac88afc: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:41.444666 6260 raft_consensus.cc:348] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f654426af8e24d208c891c8404d751cf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38479 } } peers { permanent_uuid: "8d9b956ecee449ee8efdae77dac88afc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35631 } } peers { permanent_uuid: "828bc9cc840949e792ccefb3d4909dba" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43293 } }
I20260809 02:09:41.444847 6260 raft_consensus.cc:374] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:41.444898 6260 raft_consensus.cc:729] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f654426af8e24d208c891c8404d751cf, State: Initialized, Role: FOLLOWER
I20260809 02:09:41.445016 6260 consensus_queue.cc:260] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f654426af8e24d208c891c8404d751cf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38479 } } peers { permanent_uuid: "8d9b956ecee449ee8efdae77dac88afc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35631 } } peers { permanent_uuid: "828bc9cc840949e792ccefb3d4909dba" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43293 } }
I20260809 02:09:41.445344 6263 log.cc:824] T 90b03aa578034bbb8e72143c45ac0b43 P 8d9b956ecee449ee8efdae77dac88afc: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:41.445341 6260 ts_tablet_manager.cc:1428] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 02:09:41.445338 5964 heartbeater.cc:503] Master 127.5.9.190:42801 was elected leader, sending a full tablet report...
I20260809 02:09:41.446246 6263 tablet_bootstrap.cc:492] T 90b03aa578034bbb8e72143c45ac0b43 P 8d9b956ecee449ee8efdae77dac88afc: No bootstrap required, opened a new log
I20260809 02:09:41.446337 6263 ts_tablet_manager.cc:1397] T 90b03aa578034bbb8e72143c45ac0b43 P 8d9b956ecee449ee8efdae77dac88afc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:09:41.446947 6262 raft_consensus.cc:348] T 90b03aa578034bbb8e72143c45ac0b43 P 828bc9cc840949e792ccefb3d4909dba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f654426af8e24d208c891c8404d751cf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38479 } } peers { permanent_uuid: "8d9b956ecee449ee8efdae77dac88afc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35631 } } peers { permanent_uuid: "828bc9cc840949e792ccefb3d4909dba" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43293 } }
I20260809 02:09:41.447145 6262 raft_consensus.cc:374] T 90b03aa578034bbb8e72143c45ac0b43 P 828bc9cc840949e792ccefb3d4909dba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:41.447182 6262 raft_consensus.cc:729] T 90b03aa578034bbb8e72143c45ac0b43 P 828bc9cc840949e792ccefb3d4909dba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 828bc9cc840949e792ccefb3d4909dba, State: Initialized, Role: FOLLOWER
I20260809 02:09:41.447309 6262 consensus_queue.cc:260] T 90b03aa578034bbb8e72143c45ac0b43 P 828bc9cc840949e792ccefb3d4909dba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f654426af8e24d208c891c8404d751cf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38479 } } peers { permanent_uuid: "8d9b956ecee449ee8efdae77dac88afc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35631 } } peers { permanent_uuid: "828bc9cc840949e792ccefb3d4909dba" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43293 } }
I20260809 02:09:41.447646 6262 ts_tablet_manager.cc:1428] T 90b03aa578034bbb8e72143c45ac0b43 P 828bc9cc840949e792ccefb3d4909dba: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260809 02:09:41.447652 6104 heartbeater.cc:503] Master 127.5.9.190:42801 was elected leader, sending a full tablet report...
I20260809 02:09:41.448868 6263 raft_consensus.cc:348] T 90b03aa578034bbb8e72143c45ac0b43 P 8d9b956ecee449ee8efdae77dac88afc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f654426af8e24d208c891c8404d751cf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38479 } } peers { permanent_uuid: "8d9b956ecee449ee8efdae77dac88afc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35631 } } peers { permanent_uuid: "828bc9cc840949e792ccefb3d4909dba" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43293 } }
I20260809 02:09:41.449067 6263 raft_consensus.cc:374] T 90b03aa578034bbb8e72143c45ac0b43 P 8d9b956ecee449ee8efdae77dac88afc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:41.449095 6263 raft_consensus.cc:729] T 90b03aa578034bbb8e72143c45ac0b43 P 8d9b956ecee449ee8efdae77dac88afc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d9b956ecee449ee8efdae77dac88afc, State: Initialized, Role: FOLLOWER
I20260809 02:09:41.449210 6263 consensus_queue.cc:260] T 90b03aa578034bbb8e72143c45ac0b43 P 8d9b956ecee449ee8efdae77dac88afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f654426af8e24d208c891c8404d751cf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38479 } } peers { permanent_uuid: "8d9b956ecee449ee8efdae77dac88afc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35631 } } peers { permanent_uuid: "828bc9cc840949e792ccefb3d4909dba" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43293 } }
I20260809 02:09:41.449538 6263 ts_tablet_manager.cc:1428] T 90b03aa578034bbb8e72143c45ac0b43 P 8d9b956ecee449ee8efdae77dac88afc: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 02:09:41.449548 6242 heartbeater.cc:503] Master 127.5.9.190:42801 was elected leader, sending a full tablet report...
W20260809 02:09:41.473165 6105 tablet.cc:2369] T 90b03aa578034bbb8e72143c45ac0b43 P 828bc9cc840949e792ccefb3d4909dba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:09:41.534344 5965 tablet.cc:2369] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:09:41.647984 6243 tablet.cc:2369] T 90b03aa578034bbb8e72143c45ac0b43 P 8d9b956ecee449ee8efdae77dac88afc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:09:41.821921 6265 raft_consensus.cc:482] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:09:41.822109 6265 raft_consensus.cc:504] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f654426af8e24d208c891c8404d751cf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38479 } } peers { permanent_uuid: "8d9b956ecee449ee8efdae77dac88afc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35631 } } peers { permanent_uuid: "828bc9cc840949e792ccefb3d4909dba" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43293 } }
I20260809 02:09:41.822701 6265 leader_election.cc:288] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d9b956ecee449ee8efdae77dac88afc (127.5.9.131:35631), 828bc9cc840949e792ccefb3d4909dba (127.5.9.130:43293)
I20260809 02:09:41.825989 6259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.823027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:60355 (local address 127.5.9.130:43293)
0809 02:09:41.823078 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:41.823080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:41.823181 (+ 101us) server_negotiation.cc:407] Connection header received
0809 02:09:41.823286 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:41.823287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:41.823299 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:41.823349 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:41.823550 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.824049 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.825193 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.825567 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.825692 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.825704 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:41.825855 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 02:09:41.825909 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:09:41.826000 6257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.822989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:57483 (local address 127.5.9.131:35631)
0809 02:09:41.823042 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:41.823045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:41.823183 (+ 138us) server_negotiation.cc:407] Connection header received
0809 02:09:41.823298 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:41.823300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:41.823310 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:41.823385 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:41.823587 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.824047 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.825199 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.825586 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.825692 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.825704 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:41.825897 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 02:09:41.825933 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 02:09:41.826015 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.822968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:43293 (local address 127.5.9.129:60355)
0809 02:09:41.823092 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:41.823104 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:41.823200 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:41.823383 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:41.823384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:41.823387 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:41.823476 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:41.823479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.824141 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.824142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:41.825090 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:41.825093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.825679 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.825680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:41.825769 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.825773 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:09:41.825828 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 02:09:41.825892 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":75,"thread_start_us":63,"threads_started":1}
I20260809 02:09:41.826021 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.822933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:35631 (local address 127.5.9.129:57483)
0809 02:09:41.823092 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:41.823103 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:41.823201 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:41.823419 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:41.823420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:41.823422 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:41.823494 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:41.823495 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.824145 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.824146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:41.825088 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:41.825091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.825708 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.825720 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:41.825794 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.825798 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:09:41.825874 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:09:41.825915 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":31,"thread_start_us":76,"threads_started":1}
I20260809 02:09:41.826493 6056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b03aa578034bbb8e72143c45ac0b43" candidate_uuid: "f654426af8e24d208c891c8404d751cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "828bc9cc840949e792ccefb3d4909dba" is_pre_election: true
I20260809 02:09:41.826591 6194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b03aa578034bbb8e72143c45ac0b43" candidate_uuid: "f654426af8e24d208c891c8404d751cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d9b956ecee449ee8efdae77dac88afc" is_pre_election: true
I20260809 02:09:41.826680 6056 raft_consensus.cc:2454] T 90b03aa578034bbb8e72143c45ac0b43 P 828bc9cc840949e792ccefb3d4909dba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f654426af8e24d208c891c8404d751cf in term 0.
I20260809 02:09:41.826817 6194 raft_consensus.cc:2454] T 90b03aa578034bbb8e72143c45ac0b43 P 8d9b956ecee449ee8efdae77dac88afc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f654426af8e24d208c891c8404d751cf in term 0.
I20260809 02:09:41.827044 5849 leader_election.cc:302] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 828bc9cc840949e792ccefb3d4909dba, f654426af8e24d208c891c8404d751cf; no voters:
I20260809 02:09:41.827247 6265 raft_consensus.cc:2781] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:09:41.827291 6265 raft_consensus.cc:482] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:09:41.827306 6265 raft_consensus.cc:3037] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:41.827981 6265 raft_consensus.cc:504] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f654426af8e24d208c891c8404d751cf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38479 } } peers { permanent_uuid: "8d9b956ecee449ee8efdae77dac88afc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35631 } } peers { permanent_uuid: "828bc9cc840949e792ccefb3d4909dba" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43293 } }
I20260809 02:09:41.828187 6265 leader_election.cc:288] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [CANDIDATE]: Term 1 election: Requested vote from peers 8d9b956ecee449ee8efdae77dac88afc (127.5.9.131:35631), 828bc9cc840949e792ccefb3d4909dba (127.5.9.130:43293)
I20260809 02:09:41.828408 6056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b03aa578034bbb8e72143c45ac0b43" candidate_uuid: "f654426af8e24d208c891c8404d751cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "828bc9cc840949e792ccefb3d4909dba"
I20260809 02:09:41.828439 6194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b03aa578034bbb8e72143c45ac0b43" candidate_uuid: "f654426af8e24d208c891c8404d751cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d9b956ecee449ee8efdae77dac88afc"
I20260809 02:09:41.828504 6056 raft_consensus.cc:3037] T 90b03aa578034bbb8e72143c45ac0b43 P 828bc9cc840949e792ccefb3d4909dba [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:41.828532 6194 raft_consensus.cc:3037] T 90b03aa578034bbb8e72143c45ac0b43 P 8d9b956ecee449ee8efdae77dac88afc [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:41.829269 6194 raft_consensus.cc:2454] T 90b03aa578034bbb8e72143c45ac0b43 P 8d9b956ecee449ee8efdae77dac88afc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f654426af8e24d208c891c8404d751cf in term 1.
I20260809 02:09:41.829280 6056 raft_consensus.cc:2454] T 90b03aa578034bbb8e72143c45ac0b43 P 828bc9cc840949e792ccefb3d4909dba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f654426af8e24d208c891c8404d751cf in term 1.
I20260809 02:09:41.829516 5848 leader_election.cc:302] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d9b956ecee449ee8efdae77dac88afc, f654426af8e24d208c891c8404d751cf; no voters:
I20260809 02:09:41.829674 6265 raft_consensus.cc:2781] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:41.829852 6265 raft_consensus.cc:686] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [term 1 LEADER]: Becoming Leader. State: Replica: f654426af8e24d208c891c8404d751cf, State: Running, Role: LEADER
I20260809 02:09:41.829969 6265 consensus_queue.cc:237] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f654426af8e24d208c891c8404d751cf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38479 } } peers { permanent_uuid: "8d9b956ecee449ee8efdae77dac88afc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35631 } } peers { permanent_uuid: "828bc9cc840949e792ccefb3d4909dba" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43293 } }
I20260809 02:09:41.830984 5764 catalog_manager.cc:5495] T 90b03aa578034bbb8e72143c45ac0b43 P f654426af8e24d208c891c8404d751cf reported cstate change: term changed from 0 to 1, leader changed from <none> to f654426af8e24d208c891c8404d751cf (127.5.9.129). New cstate: current_term: 1 leader_uuid: "f654426af8e24d208c891c8404d751cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f654426af8e24d208c891c8404d751cf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d9b956ecee449ee8efdae77dac88afc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "828bc9cc840949e792ccefb3d4909dba" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43293 } health_report { overall_health: UNKNOWN } } }
Aug 09 02:09:41 dist-test-slave-p3t5 krb5kdc[5717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.129@KRBTEST.COM
I20260809 02:09:41.881933 6258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.873134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56286 (local address 127.5.9.129:38479)
0809 02:09:41.873220 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:41.873223 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:41.873331 (+ 108us) server_negotiation.cc:407] Connection header received
0809 02:09:41.873414 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:41.873415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:41.873430 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:41.873509 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:41.874214 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.874717 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.875492 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.875694 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.875775 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.878562 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:41.878579 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:41.878580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:41.878588 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:41.880297 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:41.880742 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:41.880746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:41.880746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:41.880814 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:41.881163 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:41.881164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:41.881165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:41.881377 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:41.881580 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:41.881704 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 02:09:41.881817 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 02:09:41 dist-test-slave-p3t5 krb5kdc[5717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241381, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:09:41 dist-test-slave-p3t5 krb5kdc[5717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241381, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.9.129@KRBTEST.COM
I20260809 02:09:41.910022 6258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.901458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56298 (local address 127.5.9.129:38479)
0809 02:09:41.901540 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:41.901542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:41.901671 (+ 129us) server_negotiation.cc:407] Connection header received
0809 02:09:41.901762 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:41.901764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:41.901778 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:41.901856 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:41.902741 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.903242 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.904098 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.904318 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.904422 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.907161 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:41.907162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:41.907163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:41.907179 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:41.908609 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:41.909037 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:41.909040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:41.909041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:41.909106 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:41.909306 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:41.909307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:41.909308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:41.909425 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:41.909564 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:41.909800 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 02:09:41.909903 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
Aug 09 02:09:41 dist-test-slave-p3t5 krb5kdc[5717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:09:41 dist-test-slave-p3t5 krb5kdc[5717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.129@KRBTEST.COM
I20260809 02:09:41.937134 6258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:41.929113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56304 (local address 127.5.9.129:38479)
0809 02:09:41.929206 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:41.929210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:41.929314 (+ 104us) server_negotiation.cc:407] Connection header received
0809 02:09:41.929424 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:41.929425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:41.929436 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:41.929515 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:41.930198 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.930744 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.931548 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:41.931751 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:41.931837 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:41.934537 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:41.934539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:41.934539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:41.934549 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:41.936016 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:41.936406 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:41.936408 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:41.936408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:41.936478 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:41.936670 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:41.936670 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:41.936671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:41.936787 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:41.936907 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:41.936980 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 02:09:41.937054 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 02:09:41.938045 5874 tablet_service.cc:2908] Rejecting scan request for tablet 90b03aa578034bbb8e72143c45ac0b43: Uninitialized: clean time has not yet been initialized
I20260809 02:09:41.939410 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 5831
I20260809 02:09:41.947183 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 5969
I20260809 02:09:41.953943 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 6109
I20260809 02:09:41.960807 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 5731
2026-08-09T02:09:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2498 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:09:42 dist-test-slave-p3t5 krb5kdc[6286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:09:42 dist-test-slave-p3t5 krb5kdc[6286](info): set up 2 sockets
Aug 09 02:09:42 dist-test-slave-p3t5 krb5kdc[6286](info): commencing operation
krb5kdc: starting...
W20260809 02:09:43.046314 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:09:43 dist-test-slave-p3t5 krb5kdc[6286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:09:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:09:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:09:43.181344 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:44495
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:44495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:43.268852 6299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:43.269096 6299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:43.269122 6299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:43.270418 6299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:09:43.270473 6299 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 02:09:43.270488 6299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:43.270500 6299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:43.270514 6299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:09:43.270525 6299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:09:43.272171 6299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:44495
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:44495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:43.272485 6299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:43.272765 6299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:43.278092 6304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:43.278092 6307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:43.278156 6299 server_base.cc:1034] running on GCE node
W20260809 02:09:43.278122 6305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:43.278728 6299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:43.279083 6299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:43.280310 6299 hybrid_clock.cc:648] HybridClock initialized: now 1786241383280282 us; error 39 us; skew 500 ppm
Aug 09 02:09:43 dist-test-slave-p3t5 krb5kdc[6286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241383, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:43.287511 6299 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:09:43.288074 6299 webserver.cc:460] Webserver started at http://127.5.9.190:41145/ using document root <none> and password file <none>
I20260809 02:09:43.288371 6299 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:43.288425 6299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:43.288599 6299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:43.289888 6299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "9e105c6c48a449d687d847d63677307f"
format_stamp: "Formatted at 2026-08-09 02:09:43 on dist-test-slave-p3t5"
server_key: "b717441719856c0a312f8244e07b77f7"
server_key_iv: "93d5ea0d5d4f6d2af42b276c917fd40c"
server_key_version: "encryptionkey@0"
I20260809 02:09:43.290299 6299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "9e105c6c48a449d687d847d63677307f"
format_stamp: "Formatted at 2026-08-09 02:09:43 on dist-test-slave-p3t5"
server_key: "b717441719856c0a312f8244e07b77f7"
server_key_iv: "93d5ea0d5d4f6d2af42b276c917fd40c"
server_key_version: "encryptionkey@0"
I20260809 02:09:43.291855 6299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 02:09:43.292725 6315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:43.292915 6299 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:43.293022 6299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "9e105c6c48a449d687d847d63677307f"
format_stamp: "Formatted at 2026-08-09 02:09:43 on dist-test-slave-p3t5"
server_key: "b717441719856c0a312f8244e07b77f7"
server_key_iv: "93d5ea0d5d4f6d2af42b276c917fd40c"
server_key_version: "encryptionkey@0"
I20260809 02:09:43.293089 6299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:43.314932 6299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:43.317246 6299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:43.317375 6299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:43.322854 6299 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:44495
I20260809 02:09:43.322953 6381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:44495 every 8 connection(s)
I20260809 02:09:43.323321 6299 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:09:43.324137 6382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:43.326655 6382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f: Bootstrap starting.
I20260809 02:09:43.327394 6382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:43.327731 6382 log.cc:824] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:43.327998 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 6299
I20260809 02:09:43.328122 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:09:43.328289 5158 external_mini_cluster.cc:1428] Setting key 9d3d6e3d33af46201b05a86eca515ddd
I20260809 02:09:43.328600 6382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f: No bootstrap required, opened a new log
I20260809 02:09:43.331457 6382 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e105c6c48a449d687d847d63677307f" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44495 } }
I20260809 02:09:43.331620 6382 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:43.331645 6382 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e105c6c48a449d687d847d63677307f, State: Initialized, Role: FOLLOWER
I20260809 02:09:43.331902 6382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e105c6c48a449d687d847d63677307f" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44495 } }
I20260809 02:09:43.331995 6382 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:09:43.332021 6382 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:09:43.332099 6382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:43.332768 6382 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e105c6c48a449d687d847d63677307f" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44495 } }
I20260809 02:09:43.332912 6382 leader_election.cc:302] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e105c6c48a449d687d847d63677307f; no voters:
I20260809 02:09:43.333143 6382 leader_election.cc:288] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:09:43.333179 6387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:43.333316 6387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [term 1 LEADER]: Becoming Leader. State: Replica: 9e105c6c48a449d687d847d63677307f, State: Running, Role: LEADER
I20260809 02:09:43.333418 6387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e105c6c48a449d687d847d63677307f" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44495 } }
I20260809 02:09:43.333459 6382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:09:43.333817 6387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e105c6c48a449d687d847d63677307f. Latest consensus state: current_term: 1 leader_uuid: "9e105c6c48a449d687d847d63677307f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e105c6c48a449d687d847d63677307f" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44495 } } }
I20260809 02:09:43.333915 6387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [sys.catalog]: This master's current role is: LEADER
I20260809 02:09:43.333833 6388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e105c6c48a449d687d847d63677307f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e105c6c48a449d687d847d63677307f" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44495 } } }
I20260809 02:09:43.334131 6388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f [sys.catalog]: This master's current role is: LEADER
I20260809 02:09:43.334132 6392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:09:43.335412 6392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 02:09:43 dist-test-slave-p3t5 krb5kdc[6286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:43.338147 6392 catalog_manager.cc:1347] Generated new cluster ID: da1e7585b1d4491cb2241ea04fd6a3bf
I20260809 02:09:43.338222 6392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:09:43.340983 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.328873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44292 (local address 127.5.9.190:44495)
0809 02:09:43.329118 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:43.329126 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:43.329154 (+ 28us) server_negotiation.cc:407] Connection header received
0809 02:09:43.330010 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:43.330019 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:43.330079 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:43.330247 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:43.330953 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.332005 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.332851 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.333542 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.334035 (+ 493us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.336815 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:43.336819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:43.336820 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:43.336835 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:43.339145 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:43.339619 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:43.339623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:43.339624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:43.339749 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:43.339974 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:43.339975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:43.339976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:43.340322 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:43.340499 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:43.340581 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 02:09:43.340735 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":102,"threads_started":1}
I20260809 02:09:43.352458 6392 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:09:43.353315 6392 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:09:43.356694 6392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f: Generated new TSK 0
I20260809 02:09:43.357148 6392 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:09:43.393468 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:44495
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:43.482399 6409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:43.482623 6409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:43.482649 6409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:43.483799 6409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:43.483875 6409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:43.483911 6409 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:09:43.483943 6409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:09:43.485498 6409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.9.190:44495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:43.485857 6409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:43.486125 6409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:43.491640 6418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:43.491716 6416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:43.491660 6415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:43.491824 6409 server_base.cc:1034] running on GCE node
I20260809 02:09:43.492151 6409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:43.492431 6409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:43.493708 6409 hybrid_clock.cc:648] HybridClock initialized: now 1786241383493690 us; error 38 us; skew 500 ppm
Aug 09 02:09:43 dist-test-slave-p3t5 krb5kdc[6286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241383, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:43.496515 6409 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:09:43.497087 6409 webserver.cc:460] Webserver started at http://127.5.9.129:41835/ using document root <none> and password file <none>
I20260809 02:09:43.497367 6409 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:43.497453 6409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:43.497656 6409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:43.498868 6409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "46757d898ebd45dabd23dee0a9e16e5d"
format_stamp: "Formatted at 2026-08-09 02:09:43 on dist-test-slave-p3t5"
server_key: "93baa904e2129cccb9ef04bea8390b87"
server_key_iv: "425bcce153b08b549543ee118da96f28"
server_key_version: "encryptionkey@0"
I20260809 02:09:43.499269 6409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "46757d898ebd45dabd23dee0a9e16e5d"
format_stamp: "Formatted at 2026-08-09 02:09:43 on dist-test-slave-p3t5"
server_key: "93baa904e2129cccb9ef04bea8390b87"
server_key_iv: "425bcce153b08b549543ee118da96f28"
server_key_version: "encryptionkey@0"
I20260809 02:09:43.500696 6409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:09:43.501502 6425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:43.501729 6409 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:43.501842 6409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "46757d898ebd45dabd23dee0a9e16e5d"
format_stamp: "Formatted at 2026-08-09 02:09:43 on dist-test-slave-p3t5"
server_key: "93baa904e2129cccb9ef04bea8390b87"
server_key_iv: "425bcce153b08b549543ee118da96f28"
server_key_version: "encryptionkey@0"
I20260809 02:09:43.501922 6409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:43.533130 6409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:43.535694 6409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:43.535871 6409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:43.536525 6409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:43.536594 6409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:43.536638 6409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:43.536667 6409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:43.544227 6409 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:38013
I20260809 02:09:43.544317 6545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:38013 every 8 connection(s)
Aug 09 02:09:43 dist-test-slave-p3t5 krb5kdc[6286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241383, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:43.544646 6409 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:09:43.548000 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.537066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:54859 (local address 127.5.9.190:44495)
0809 02:09:43.537127 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:43.537129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:43.537387 (+ 258us) server_negotiation.cc:407] Connection header received
0809 02:09:43.538045 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:43.538046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:43.538060 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:43.538111 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:43.539167 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.540074 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.541750 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.542110 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.542211 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.545241 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:43.545243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:43.545243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:43.545251 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:43.546679 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:43.547093 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:43.547094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:43.547094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:43.547150 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:43.547383 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:43.547384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:43.547384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:43.547500 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:43.547552 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:43.547840 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 02:09:43.547930 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 02:09:43.548117 6434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.537057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:44495 (local address 127.5.9.129:54859)
0809 02:09:43.537256 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:43.537296 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:43.537924 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:43.538215 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:43.538220 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:43.538648 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:43.539022 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.539029 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.540193 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.540195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.541645 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.541650 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.542246 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.542247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.542403 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.543077 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:43.543090 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:43.545133 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:43.546777 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:43.546783 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:43.546785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:43.547026 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:43.547210 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:43.547211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:43.547211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:43.547327 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:43.547556 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:43.547561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:43.547685 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 02:09:43.547771 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 02:09:43.547906 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":101,"threads_started":1}
I20260809 02:09:43.548676 6546 heartbeater.cc:347] Connected to a master server at 127.5.9.190:44495
I20260809 02:09:43.548779 6546 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:43.548995 6546 heartbeater.cc:511] Master 127.5.9.190:44495 requested a full tablet report, sending...
I20260809 02:09:43.549633 6331 ts_manager.cc:194] Registered new tserver with Master: 46757d898ebd45dabd23dee0a9e16e5d (127.5.9.129:38013)
I20260809 02:09:43.550616 6331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:54859
I20260809 02:09:43.550909 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 6409
I20260809 02:09:43.551017 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:09:43.551192 5158 external_mini_cluster.cc:1428] Setting key b990832ec838b6e693c52e94821321ad
I20260809 02:09:43.556870 6552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.549876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:44495 (local address 127.5.9.129:35671)
0809 02:09:43.550062 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:43.550074 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:43.550140 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:43.550261 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:43.550262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:43.550476 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:43.550542 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.550544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.551532 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.551533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.552298 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.552302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.552652 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.552653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.552736 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.553260 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:43.553265 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:43.553758 (+ 493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:43.555709 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:43.555711 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:43.555711 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:43.555906 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:43.556156 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:43.556157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:43.556158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:43.556233 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:43.556579 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:43.556580 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:43.556628 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 02:09:43.556690 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 02:09:43.556784 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":116,"threads_started":1}
I20260809 02:09:43.556962 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.549988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:35671 (local address 127.5.9.190:44495)
0809 02:09:43.550040 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:43.550042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:43.550126 (+ 84us) server_negotiation.cc:407] Connection header received
0809 02:09:43.550196 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:43.550196 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:43.550205 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:43.550256 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:43.550633 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.551426 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.552392 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.552575 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.552631 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.553854 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:43.553856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:43.553856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:43.553865 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:43.555604 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:43.555990 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:43.555991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:43.555992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:43.556064 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:43.556318 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:43.556319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:43.556320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:43.556489 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:43.556575 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:43.556745 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 02:09:43.556839 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:09:43.582588 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:44495
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:43.669742 6555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:43.669981 6555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:43.670007 6555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:43.671276 6555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:43.671334 6555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:43.671356 6555 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:09:43.671375 6555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:09:43.672914 6555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.9.190:44495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:43.673194 6555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:43.673457 6555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:43.679014 6563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:43.679010 6561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:43.679019 6560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:43.679306 6555 server_base.cc:1034] running on GCE node
I20260809 02:09:43.679513 6555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:43.679728 6555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:43.680920 6555 hybrid_clock.cc:648] HybridClock initialized: now 1786241383680885 us; error 45 us; skew 500 ppm
Aug 09 02:09:43 dist-test-slave-p3t5 krb5kdc[6286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241383, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:43.683640 6555 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:09:43.684190 6555 webserver.cc:460] Webserver started at http://127.5.9.130:41361/ using document root <none> and password file <none>
I20260809 02:09:43.684514 6555 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:43.684589 6555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:43.684765 6555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:43.686081 6555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da"
format_stamp: "Formatted at 2026-08-09 02:09:43 on dist-test-slave-p3t5"
server_key: "f0db0d444052c746367323c1deceb765"
server_key_iv: "bca44a0db3b07f8c576de2b34e110481"
server_key_version: "encryptionkey@0"
I20260809 02:09:43.686479 6555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da"
format_stamp: "Formatted at 2026-08-09 02:09:43 on dist-test-slave-p3t5"
server_key: "f0db0d444052c746367323c1deceb765"
server_key_iv: "bca44a0db3b07f8c576de2b34e110481"
server_key_version: "encryptionkey@0"
I20260809 02:09:43.688081 6555 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:09:43.688894 6570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:43.689087 6555 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:43.689200 6555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da"
format_stamp: "Formatted at 2026-08-09 02:09:43 on dist-test-slave-p3t5"
server_key: "f0db0d444052c746367323c1deceb765"
server_key_iv: "bca44a0db3b07f8c576de2b34e110481"
server_key_version: "encryptionkey@0"
I20260809 02:09:43.689285 6555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:43.709820 6555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:43.712102 6555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:43.712298 6555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:43.712985 6555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:43.713042 6555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:43.713078 6555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:43.713101 6555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:09:43 dist-test-slave-p3t5 krb5kdc[6286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241383, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:43.720853 6555 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:34701
I20260809 02:09:43.720929 6693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:34701 every 8 connection(s)
I20260809 02:09:43.721261 6555 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:09:43.724543 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.713543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:35751 (local address 127.5.9.190:44495)
0809 02:09:43.713646 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:43.713649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:43.713880 (+ 231us) server_negotiation.cc:407] Connection header received
0809 02:09:43.714516 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:43.714517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:43.714531 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:43.714617 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:43.715366 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.715947 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.717172 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.717415 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.717521 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.721284 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:43.721287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:43.721287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:43.721305 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:43.722958 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:43.723453 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:43.723454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:43.723455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:43.723531 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:43.723816 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:43.723817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:43.723817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:43.723960 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:43.724038 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:43.724343 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 02:09:43.724445 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:09:43.724778 6579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.713501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:44495 (local address 127.5.9.130:35751)
0809 02:09:43.713717 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:43.713766 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:43.714398 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:43.714697 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:43.714701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:43.715021 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:43.715238 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.715243 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.716073 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.716075 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.717057 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.717059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.717536 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.717538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.717762 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.718469 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:43.718479 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:43.721154 (+ 2675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:43.723087 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:43.723094 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:43.723096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:43.723357 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:43.723613 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:43.723614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:43.723614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:43.723735 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:43.724041 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:43.724049 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:43.724203 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 02:09:43.724321 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 02:09:43.724500 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":126,"threads_started":1}
I20260809 02:09:43.725345 6694 heartbeater.cc:347] Connected to a master server at 127.5.9.190:44495
I20260809 02:09:43.725435 6694 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:43.725693 6694 heartbeater.cc:511] Master 127.5.9.190:44495 requested a full tablet report, sending...
I20260809 02:09:43.726162 6332 ts_manager.cc:194] Registered new tserver with Master: f2b8dbbab98f4b2f9aff6ff6f2f855da (127.5.9.130:34701)
I20260809 02:09:43.726699 6332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:35751
I20260809 02:09:43.729152 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 6555
I20260809 02:09:43.729244 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:09:43.729460 5158 external_mini_cluster.cc:1428] Setting key daf1276e6a78ed6c1c5909ebf4e49d4f
I20260809 02:09:43.732903 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.726650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:56803 (local address 127.5.9.190:44495)
0809 02:09:43.726695 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:43.726697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:43.726773 (+ 76us) server_negotiation.cc:407] Connection header received
0809 02:09:43.726879 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:43.726880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:43.726892 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:43.726950 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:43.727339 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.727901 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.728615 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.728801 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.728868 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.729990 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:43.729991 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:43.729992 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:43.729999 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:43.731669 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:43.732031 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:43.732033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:43.732033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:43.732105 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:43.732368 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:43.732368 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:43.732369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:43.732533 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:43.732591 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:43.732769 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 02:09:43.732837 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:09:43.733043 6701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.726570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:44495 (local address 127.5.9.130:56803)
0809 02:09:43.726702 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:43.726714 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:43.726792 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:43.726961 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:43.726962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:43.727193 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:43.727256 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.727258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.727994 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.727995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.728523 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.728524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.728880 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.728881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.728937 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.729447 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:43.729452 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:43.729909 (+ 457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:43.731766 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:43.731767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:43.731767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:43.731955 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:43.732198 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:43.732198 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:43.732199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:43.732285 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:43.732604 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:43.732605 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:43.732661 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 02:09:43.732749 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 02:09:43.732893 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:09:43.759045 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:44495
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:43.847357 6704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:43.847661 6704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:43.847707 6704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:43.849115 6704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:43.849202 6704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:43.849236 6704 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:09:43.849274 6704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:09:43.850945 6704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.9.190:44495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:43.851285 6704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:43.851558 6704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:43.856932 6709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:43.856938 6712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:43.856938 6710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:43.857168 6704 server_base.cc:1034] running on GCE node
I20260809 02:09:43.857471 6704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:43.857805 6704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:43.858963 6704 hybrid_clock.cc:648] HybridClock initialized: now 1786241383858946 us; error 31 us; skew 500 ppm
Aug 09 02:09:43 dist-test-slave-p3t5 krb5kdc[6286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241383, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:43.861726 6704 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:09:43.862236 6704 webserver.cc:460] Webserver started at http://127.5.9.131:41077/ using document root <none> and password file <none>
I20260809 02:09:43.862462 6704 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:43.862516 6704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:43.862615 6704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:43.863703 6704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "00bacf201b694fea8e34e9e7d18000cc"
format_stamp: "Formatted at 2026-08-09 02:09:43 on dist-test-slave-p3t5"
server_key: "915b1d7ae920a961a53cbff48b705091"
server_key_iv: "7f5f23a64aaaf8800ec3bc2e60428786"
server_key_version: "encryptionkey@0"
I20260809 02:09:43.864045 6704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "00bacf201b694fea8e34e9e7d18000cc"
format_stamp: "Formatted at 2026-08-09 02:09:43 on dist-test-slave-p3t5"
server_key: "915b1d7ae920a961a53cbff48b705091"
server_key_iv: "7f5f23a64aaaf8800ec3bc2e60428786"
server_key_version: "encryptionkey@0"
I20260809 02:09:43.865482 6704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:09:43.866456 6719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:43.866675 6704 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:43.866766 6704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "00bacf201b694fea8e34e9e7d18000cc"
format_stamp: "Formatted at 2026-08-09 02:09:43 on dist-test-slave-p3t5"
server_key: "915b1d7ae920a961a53cbff48b705091"
server_key_iv: "7f5f23a64aaaf8800ec3bc2e60428786"
server_key_version: "encryptionkey@0"
I20260809 02:09:43.866829 6704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:43.895869 6704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:43.899003 6704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:43.899205 6704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:43.900010 6704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:43.900074 6704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:43.900112 6704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:43.900133 6704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:43.908495 6704 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:45965
I20260809 02:09:43.908902 6704 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
Aug 09 02:09:43 dist-test-slave-p3t5 krb5kdc[6286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241383, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:43.909143 6837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:45965 every 8 connection(s)
I20260809 02:09:43.913811 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.900320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:52491 (local address 127.5.9.190:44495)
0809 02:09:43.900373 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:43.900375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:43.900986 (+ 611us) server_negotiation.cc:407] Connection header received
0809 02:09:43.902296 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:43.902298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:43.902315 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:43.902352 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:43.903446 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.904285 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.906087 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.906421 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.906482 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.910138 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:43.910141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:43.910142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:43.910153 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:43.912014 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:43.912658 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:43.912660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:43.912660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:43.912740 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:43.913010 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:43.913011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:43.913011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:43.913180 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:43.913254 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:43.913562 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 02:09:43.913701 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:09:43.913978 6728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.900535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:44495 (local address 127.5.9.131:52491)
0809 02:09:43.900838 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:43.900888 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:43.902145 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:43.902586 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:43.902591 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:43.902989 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:43.903311 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.903319 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.904429 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.904430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.905956 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.905959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.906598 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.906599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.906791 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.907457 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:43.907478 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:43.910000 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:43.912151 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:43.912161 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:43.912164 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:43.912550 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:43.912809 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:43.912809 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:43.912810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:43.912926 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:43.913263 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:43.913268 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:43.913405 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 02:09:43.913488 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 02:09:43.913688 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":102,"threads_started":1}
I20260809 02:09:43.914762 6838 heartbeater.cc:347] Connected to a master server at 127.5.9.190:44495
I20260809 02:09:43.914887 6838 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:43.915091 6838 heartbeater.cc:511] Master 127.5.9.190:44495 requested a full tablet report, sending...
I20260809 02:09:43.915498 6332 ts_manager.cc:194] Registered new tserver with Master: 00bacf201b694fea8e34e9e7d18000cc (127.5.9.131:45965)
I20260809 02:09:43.915786 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 6704
I20260809 02:09:43.915886 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:09:43.916061 5158 external_mini_cluster.cc:1428] Setting key bb713750c30a834b8f1695dea15a7abb
I20260809 02:09:43.916043 6332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:52491
I20260809 02:09:43.917658 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:09:43.922832 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.915984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:44611 (local address 127.5.9.190:44495)
0809 02:09:43.916020 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:43.916022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:43.916138 (+ 116us) server_negotiation.cc:407] Connection header received
0809 02:09:43.916253 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:43.916255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:43.916270 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:43.916335 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:43.916771 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.917340 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.918296 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.918583 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.918661 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.919712 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:43.919713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:43.919714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:43.919722 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:43.921457 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:43.921941 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:43.921943 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:43.921943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:43.922022 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:43.922241 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:43.922243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:43.922243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:43.922384 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:43.922465 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:43.922631 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 02:09:43.922728 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 02:09:43.922859 6845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.915824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:44495 (local address 127.5.9.131:44611)
0809 02:09:43.916078 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:43.916089 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:43.916163 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:43.916342 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:43.916343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:43.916595 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:43.916686 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.916689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.917492 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.917493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.918200 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.918203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.918666 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.918667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.918750 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.919287 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:43.919294 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:43.919616 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:43.921551 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:43.921552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:43.921552 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:43.921847 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:43.922098 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:43.922099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:43.922099 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:43.922168 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:43.922458 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:43.922459 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:43.922516 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 02:09:43.922598 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 02:09:43.922739 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":13,"thread_start_us":74,"threads_started":1}
Aug 09 02:09:43 dist-test-slave-p3t5 krb5kdc[6286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241383, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:09:43 dist-test-slave-p3t5 krb5kdc[6286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241383, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:43.945274 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.937686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44304 (local address 127.5.9.190:44495)
0809 02:09:43.937762 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:43.937765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:43.937788 (+ 23us) server_negotiation.cc:407] Connection header received
0809 02:09:43.937850 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:43.937855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:43.937866 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:43.937936 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:43.938561 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.939087 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.939868 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.940075 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.940157 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.942668 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:43.942670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:43.942670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:43.942680 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:43.943967 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:43.944386 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:43.944387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:43.944388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:43.944455 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:43.944669 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:43.944670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:43.944671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:43.944835 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:43.944953 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:43.945067 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 02:09:43.945154 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 02:09:43.947009 6332 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:09:43.948841 6332 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:09:43.956125 6858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.951508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:45965 (local address 127.0.0.1:51394)
0809 02:09:43.951889 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:43.951899 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:43.951955 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:43.952832 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:43.952833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:43.952835 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:43.952872 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.952873 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.954106 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.954107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.954968 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.954971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.955755 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.955756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.955847 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.955862 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 02:09:43.955948 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 02:09:43.956012 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":61,"threads_started":1}
I20260809 02:09:43.956238 6859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.951700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48838 (local address 127.5.9.130:34701)
0809 02:09:43.952407 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:43.952412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:43.952423 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:09:43.952460 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:43.952462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:43.952476 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:43.952549 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:43.952720 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.953908 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.955171 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.955870 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.955941 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.955993 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:43.956089 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 02:09:43.956130 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":100,"threads_started":1}
I20260809 02:09:43.956269 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.951518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:38013 (local address 127.0.0.1:47626)
0809 02:09:43.951920 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:43.951932 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:43.951992 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:43.952285 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:43.952288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:43.952290 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:43.952378 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.952380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.953494 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.953495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.954837 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.954840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.955936 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.955937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.956021 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.956025 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:09:43.956080 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 02:09:43.956130 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":120,"threads_started":1}
I20260809 02:09:43.956542 6860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.951705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51394 (local address 127.5.9.131:45965)
0809 02:09:43.952648 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:43.952651 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:43.952659 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:09:43.952716 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:43.952718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:43.952732 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:43.952825 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:43.952939 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.954012 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.955064 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.955655 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.955754 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.955824 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:43.956383 (+ 559us) server_negotiation.cc:299] Negotiation successful
0809 02:09:43.956439 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":914,"thread_start_us":108,"threads_started":1}
I20260809 02:09:43.956542 6861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.951851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47626 (local address 127.5.9.129:38013)
0809 02:09:43.952079 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:43.952085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:43.952095 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:09:43.952140 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:43.952142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:43.952158 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:43.952257 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:43.952453 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.953404 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.954946 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.955753 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.955831 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.955896 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:43.956385 (+ 489us) server_negotiation.cc:299] Negotiation successful
0809 02:09:43.956438 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":86,"threads_started":1}
I20260809 02:09:43.957100 6856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.951508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:34701 (local address 127.0.0.1:48838)
0809 02:09:43.951812 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:43.951863 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:43.951948 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:43.952569 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:43.952570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:43.952572 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:43.952627 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.952629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.954006 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.954007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.955083 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.955086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.955957 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.955957 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.956015 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.956018 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:09:43.956075 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 02:09:43.956111 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"spinlock_wait_cycles":384,"thread_start_us":118,"threads_started":1}
I20260809 02:09:43.957257 6770 tablet_service.cc:1467] Processing CreateTablet for tablet d27d97a8ed0f4bfbbe20708448749b1d (DEFAULT_TABLE table=test-table [id=a163448919c74a8eb1ea8023c6a3e09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:43.957417 6476 tablet_service.cc:1467] Processing CreateTablet for tablet d27d97a8ed0f4bfbbe20708448749b1d (DEFAULT_TABLE table=test-table [id=a163448919c74a8eb1ea8023c6a3e09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:43.957800 6770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d27d97a8ed0f4bfbbe20708448749b1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:43.957935 6476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d27d97a8ed0f4bfbbe20708448749b1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:43.957917 6623 tablet_service.cc:1467] Processing CreateTablet for tablet d27d97a8ed0f4bfbbe20708448749b1d (DEFAULT_TABLE table=test-table [id=a163448919c74a8eb1ea8023c6a3e09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:43.958240 6623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d27d97a8ed0f4bfbbe20708448749b1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:43.961033 6862 tablet_bootstrap.cc:492] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d: Bootstrap starting.
I20260809 02:09:43.961772 6863 tablet_bootstrap.cc:492] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc: Bootstrap starting.
I20260809 02:09:43.961925 6862 tablet_bootstrap.cc:654] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:43.962052 6864 tablet_bootstrap.cc:492] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da: Bootstrap starting.
I20260809 02:09:43.962297 6862 log.cc:824] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:43.962770 6863 tablet_bootstrap.cc:654] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:43.962770 6864 tablet_bootstrap.cc:654] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:43.963124 6862 tablet_bootstrap.cc:492] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d: No bootstrap required, opened a new log
I20260809 02:09:43.963142 6864 log.cc:824] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:43.963143 6863 log.cc:824] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:43.963217 6862 ts_tablet_manager.cc:1397] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:09:43.963963 6864 tablet_bootstrap.cc:492] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da: No bootstrap required, opened a new log
I20260809 02:09:43.964039 6864 ts_tablet_manager.cc:1397] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:09:43.964185 6863 tablet_bootstrap.cc:492] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc: No bootstrap required, opened a new log
I20260809 02:09:43.964249 6863 ts_tablet_manager.cc:1397] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:09:43.965196 6862 raft_consensus.cc:348] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } }
I20260809 02:09:43.965392 6862 raft_consensus.cc:374] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:43.965454 6862 raft_consensus.cc:729] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46757d898ebd45dabd23dee0a9e16e5d, State: Initialized, Role: FOLLOWER
I20260809 02:09:43.965579 6862 consensus_queue.cc:260] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } }
I20260809 02:09:43.965899 6862 ts_tablet_manager.cc:1428] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:09:43.966058 6546 heartbeater.cc:503] Master 127.5.9.190:44495 was elected leader, sending a full tablet report...
I20260809 02:09:43.966153 6864 raft_consensus.cc:348] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } }
I20260809 02:09:43.966326 6864 raft_consensus.cc:374] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:43.966362 6864 raft_consensus.cc:729] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b8dbbab98f4b2f9aff6ff6f2f855da, State: Initialized, Role: FOLLOWER
I20260809 02:09:43.966271 6863 raft_consensus.cc:348] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } }
I20260809 02:09:43.966426 6863 raft_consensus.cc:374] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:43.966516 6863 raft_consensus.cc:729] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00bacf201b694fea8e34e9e7d18000cc, State: Initialized, Role: FOLLOWER
I20260809 02:09:43.966478 6864 consensus_queue.cc:260] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } }
I20260809 02:09:43.966656 6863 consensus_queue.cc:260] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } }
I20260809 02:09:43.966789 6864 ts_tablet_manager.cc:1428] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:09:43.966857 6694 heartbeater.cc:503] Master 127.5.9.190:44495 was elected leader, sending a full tablet report...
I20260809 02:09:43.966946 6863 ts_tablet_manager.cc:1428] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:09:43.967082 6838 heartbeater.cc:503] Master 127.5.9.190:44495 was elected leader, sending a full tablet report...
I20260809 02:09:43.967798 6868 raft_consensus.cc:482] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:09:43.967890 6868 raft_consensus.cc:504] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } }
I20260809 02:09:43.968245 6868 leader_election.cc:288] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2b8dbbab98f4b2f9aff6ff6f2f855da (127.5.9.130:34701), 00bacf201b694fea8e34e9e7d18000cc (127.5.9.131:45965)
I20260809 02:09:43.971452 6434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.968414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:34701 (local address 127.5.9.129:45575)
0809 02:09:43.968453 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:43.968467 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:43.968512 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:43.968689 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:43.968690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:43.968691 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:43.968763 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.968765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.969605 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.969606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.970597 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.970600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.971159 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.971160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.971253 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.971256 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:09:43.971325 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 02:09:43.971369 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 02:09:43.971452 6859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.968548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:45575 (local address 127.5.9.130:34701)
0809 02:09:43.968588 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:43.968589 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:43.968596 (+ 7us) server_negotiation.cc:407] Connection header received
0809 02:09:43.968618 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:43.968619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:43.968627 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:43.968673 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:43.968841 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.969506 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.970689 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.971061 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.971156 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.971168 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:43.971331 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 02:09:43.971372 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:09:43.971546 6860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.968446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:56569 (local address 127.5.9.131:45965)
0809 02:09:43.968506 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:43.968508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:43.968711 (+ 203us) server_negotiation.cc:407] Connection header received
0809 02:09:43.968775 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:43.968776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:43.968787 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:43.968899 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:43.969129 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.969790 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.970803 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.971179 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.971265 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.971276 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:43.971452 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 02:09:43.971488 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:09:43.971863 6646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d27d97a8ed0f4bfbbe20708448749b1d" candidate_uuid: "46757d898ebd45dabd23dee0a9e16e5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" is_pre_election: true
W20260809 02:09:43.971975 6695 tablet.cc:2369] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:09:43.972020 6646 raft_consensus.cc:2454] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46757d898ebd45dabd23dee0a9e16e5d in term 0.
I20260809 02:09:43.972064 6871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.968418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:45965 (local address 127.5.9.129:56569)
0809 02:09:43.968627 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:43.968637 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:43.968724 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:43.968900 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:43.968901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:43.968903 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:43.969053 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.969054 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.969883 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.969884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.970700 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:43.970703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.971267 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.971268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:43.971365 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.971369 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:09:43.971433 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 02:09:43.971478 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":57,"thread_start_us":106,"threads_started":1}
I20260809 02:09:43.972267 6426 leader_election.cc:302] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46757d898ebd45dabd23dee0a9e16e5d, f2b8dbbab98f4b2f9aff6ff6f2f855da; no voters:
I20260809 02:09:43.972435 6868 raft_consensus.cc:2781] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:09:43.972517 6868 raft_consensus.cc:482] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:09:43.972548 6868 raft_consensus.cc:3037] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:43.972636 6789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d27d97a8ed0f4bfbbe20708448749b1d" candidate_uuid: "46757d898ebd45dabd23dee0a9e16e5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00bacf201b694fea8e34e9e7d18000cc" is_pre_election: true
I20260809 02:09:43.972781 6789 raft_consensus.cc:2454] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46757d898ebd45dabd23dee0a9e16e5d in term 0.
I20260809 02:09:43.973805 6868 raft_consensus.cc:504] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } }
I20260809 02:09:43.973961 6868 leader_election.cc:288] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [CANDIDATE]: Term 1 election: Requested vote from peers f2b8dbbab98f4b2f9aff6ff6f2f855da (127.5.9.130:34701), 00bacf201b694fea8e34e9e7d18000cc (127.5.9.131:45965)
I20260809 02:09:43.974121 6789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d27d97a8ed0f4bfbbe20708448749b1d" candidate_uuid: "46757d898ebd45dabd23dee0a9e16e5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00bacf201b694fea8e34e9e7d18000cc"
I20260809 02:09:43.974121 6646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d27d97a8ed0f4bfbbe20708448749b1d" candidate_uuid: "46757d898ebd45dabd23dee0a9e16e5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da"
I20260809 02:09:43.974213 6789 raft_consensus.cc:3037] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:43.974215 6646 raft_consensus.cc:3037] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:43.975171 6646 raft_consensus.cc:2454] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46757d898ebd45dabd23dee0a9e16e5d in term 1.
I20260809 02:09:43.975171 6789 raft_consensus.cc:2454] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46757d898ebd45dabd23dee0a9e16e5d in term 1.
I20260809 02:09:43.975358 6427 leader_election.cc:302] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00bacf201b694fea8e34e9e7d18000cc, 46757d898ebd45dabd23dee0a9e16e5d; no voters:
I20260809 02:09:43.975492 6868 raft_consensus.cc:2781] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:43.975660 6868 raft_consensus.cc:686] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [term 1 LEADER]: Becoming Leader. State: Replica: 46757d898ebd45dabd23dee0a9e16e5d, State: Running, Role: LEADER
I20260809 02:09:43.975800 6868 consensus_queue.cc:237] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } }
I20260809 02:09:43.976691 6332 catalog_manager.cc:5495] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d reported cstate change: term changed from 0 to 1, leader changed from <none> to 46757d898ebd45dabd23dee0a9e16e5d (127.5.9.129). New cstate: current_term: 1 leader_uuid: "46757d898ebd45dabd23dee0a9e16e5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } health_report { overall_health: HEALTHY } } }
I20260809 02:09:43.987970 6861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:43.985081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47642 (local address 127.5.9.129:38013)
0809 02:09:43.985152 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:43.985154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:43.985164 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:09:43.985269 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:43.985270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:43.985290 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:43.985359 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:09:43.985537 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.986117 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.986943 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:43.987151 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:43.987224 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:43.987502 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:09:43.987646 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:09:43.987721 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:43.987792 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 02:09:43.987819 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:09:43.990127 6646 raft_consensus.cc:1261] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 1 FOLLOWER]: Refusing update from remote peer 46757d898ebd45dabd23dee0a9e16e5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:09:43.990127 6789 raft_consensus.cc:1261] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc [term 1 FOLLOWER]: Refusing update from remote peer 46757d898ebd45dabd23dee0a9e16e5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:09:43.990638 6872 consensus_queue.cc:1035] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:09:43.990794 6868 consensus_queue.cc:1035] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:09:43.993685 6877 mvcc.cc:205] Tried to move back new op lower bound from 7316444708820180992 to 7316444708765503488. Current Snapshot: MvccSnapshot[applied={T|T < 7316444708820180992}]
I20260809 02:09:43.997670 6332 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:09:43.997838 6332 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:09:43.998802 6332 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f: Sending DeleteTablet for 3 replicas of tablet d27d97a8ed0f4bfbbe20708448749b1d
I20260809 02:09:43.999262 6623 tablet_service.cc:1514] Processing DeleteTablet for tablet d27d97a8ed0f4bfbbe20708448749b1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:43 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:48838
I20260809 02:09:43.999298 6770 tablet_service.cc:1514] Processing DeleteTablet for tablet d27d97a8ed0f4bfbbe20708448749b1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:43 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:51394
I20260809 02:09:43.999329 6766 tablet_service.cc:1514] Processing DeleteTablet for tablet d27d97a8ed0f4bfbbe20708448749b1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:43 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:51394
I20260809 02:09:43.999512 6476 tablet_service.cc:1514] Processing DeleteTablet for tablet d27d97a8ed0f4bfbbe20708448749b1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:43 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:47626
I20260809 02:09:43.999776 6887 tablet_replica.cc:330] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d: stopping tablet replica
I20260809 02:09:43.999768 6332 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:09:43.999936 6332 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:09:44.000018 6885 tablet_replica.cc:330] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da: stopping tablet replica
I20260809 02:09:44.000133 6885 raft_consensus.cc:2229] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:09:44.000146 6887 raft_consensus.cc:2229] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:09:44.000272 6885 raft_consensus.cc:2258] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:44.000414 6887 raft_consensus.cc:2258] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:44.000823 6885 ts_tablet_manager.cc:1905] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:44.001003 6887 ts_tablet_manager.cc:1905] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:44.001375 6476 tablet_service.cc:1467] Processing CreateTablet for tablet 0d55790897514affa0ba150d6ff65aec (DEFAULT_TABLE table=test-table [id=5d84f226cc754718b652a8ff98baeaf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:44.001482 6770 tablet_service.cc:1467] Processing CreateTablet for tablet 0d55790897514affa0ba150d6ff65aec (DEFAULT_TABLE table=test-table [id=5d84f226cc754718b652a8ff98baeaf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:44.001510 6476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d55790897514affa0ba150d6ff65aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:44.001602 6770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d55790897514affa0ba150d6ff65aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:44.003015 6862 tablet_bootstrap.cc:492] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d: Bootstrap starting.
I20260809 02:09:44.003024 6863 tablet_bootstrap.cc:492] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc: Bootstrap starting.
I20260809 02:09:44.003087 6623 tablet_service.cc:1467] Processing CreateTablet for tablet 0d55790897514affa0ba150d6ff65aec (DEFAULT_TABLE table=test-table [id=5d84f226cc754718b652a8ff98baeaf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:44.003129 6887 ts_tablet_manager.cc:1918] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:09:44.003209 6623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d55790897514affa0ba150d6ff65aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:44.003330 6887 log.cc:1196] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/wal/wals/d27d97a8ed0f4bfbbe20708448749b1d
I20260809 02:09:44.003454 6862 tablet_bootstrap.cc:654] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:44.003657 6863 tablet_bootstrap.cc:654] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:44.003919 6887 ts_tablet_manager.cc:1939] T d27d97a8ed0f4bfbbe20708448749b1d P 46757d898ebd45dabd23dee0a9e16e5d: Deleting consensus metadata
I20260809 02:09:44.004431 6318 catalog_manager.cc:4841] TS 46757d898ebd45dabd23dee0a9e16e5d (127.5.9.129:38013): tablet d27d97a8ed0f4bfbbe20708448749b1d (table test-table [id=a163448919c74a8eb1ea8023c6a3e09f]) successfully deleted
I20260809 02:09:44.005124 6864 tablet_bootstrap.cc:492] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da: Bootstrap starting.
I20260809 02:09:44.005688 6864 tablet_bootstrap.cc:654] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:44.006510 6864 tablet_bootstrap.cc:492] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da: No bootstrap required, opened a new log
I20260809 02:09:44.006604 6864 ts_tablet_manager.cc:1397] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:44.006788 6864 raft_consensus.cc:348] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.006883 6864 raft_consensus.cc:374] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:44.006907 6864 raft_consensus.cc:729] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b8dbbab98f4b2f9aff6ff6f2f855da, State: Initialized, Role: FOLLOWER
I20260809 02:09:44.006974 6864 consensus_queue.cc:260] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.007014 6886 tablet_replica.cc:330] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc: stopping tablet replica
I20260809 02:09:44.007069 6864 ts_tablet_manager.cc:1428] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:09:44.007145 6886 raft_consensus.cc:2229] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:09:44.007282 6886 raft_consensus.cc:2258] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:44.006907 6885 ts_tablet_manager.cc:1918] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:09:44.007485 6885 log.cc:1196] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/d27d97a8ed0f4bfbbe20708448749b1d
I20260809 02:09:44.007762 6885 ts_tablet_manager.cc:1939] T d27d97a8ed0f4bfbbe20708448749b1d P f2b8dbbab98f4b2f9aff6ff6f2f855da: Deleting consensus metadata
I20260809 02:09:44.008237 6316 catalog_manager.cc:4841] TS f2b8dbbab98f4b2f9aff6ff6f2f855da (127.5.9.130:34701): tablet d27d97a8ed0f4bfbbe20708448749b1d (table test-table [id=a163448919c74a8eb1ea8023c6a3e09f]) successfully deleted
I20260809 02:09:44.008597 6863 tablet_bootstrap.cc:492] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc: No bootstrap required, opened a new log
I20260809 02:09:44.008653 6863 ts_tablet_manager.cc:1397] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260809 02:09:44.008682 6886 ts_tablet_manager.cc:1905] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:44.008803 6863 raft_consensus.cc:348] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.008881 6863 raft_consensus.cc:374] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:44.008906 6863 raft_consensus.cc:729] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00bacf201b694fea8e34e9e7d18000cc, State: Initialized, Role: FOLLOWER
I20260809 02:09:44.008961 6863 consensus_queue.cc:260] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.009040 6863 ts_tablet_manager.cc:1428] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:09:44.010169 6886 ts_tablet_manager.cc:1918] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:09:44.010236 6886 log.cc:1196] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/d27d97a8ed0f4bfbbe20708448749b1d
I20260809 02:09:44.010434 6886 ts_tablet_manager.cc:1939] T d27d97a8ed0f4bfbbe20708448749b1d P 00bacf201b694fea8e34e9e7d18000cc: Deleting consensus metadata
I20260809 02:09:44.010869 6317 catalog_manager.cc:4841] TS 00bacf201b694fea8e34e9e7d18000cc (127.5.9.131:45965): tablet d27d97a8ed0f4bfbbe20708448749b1d (table test-table [id=a163448919c74a8eb1ea8023c6a3e09f]) successfully deleted
W20260809 02:09:44.011071 6317 catalog_manager.cc:4808] TS 00bacf201b694fea8e34e9e7d18000cc (127.5.9.131:45965): delete failed for tablet d27d97a8ed0f4bfbbe20708448749b1d because the tablet was not found. No further retry: Not found: Tablet not found: d27d97a8ed0f4bfbbe20708448749b1d
I20260809 02:09:44.015180 6862 tablet_bootstrap.cc:492] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d: No bootstrap required, opened a new log
I20260809 02:09:44.015300 6862 ts_tablet_manager.cc:1397] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20260809 02:09:44.015503 6862 raft_consensus.cc:348] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.015933 6862 raft_consensus.cc:374] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:44.015971 6862 raft_consensus.cc:729] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46757d898ebd45dabd23dee0a9e16e5d, State: Initialized, Role: FOLLOWER
I20260809 02:09:44.016026 6862 consensus_queue.cc:260] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.016144 6862 ts_tablet_manager.cc:1428] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:44.025370 6870 raft_consensus.cc:482] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:09:44.025506 6870 raft_consensus.cc:504] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.025954 6870 leader_election.cc:288] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46757d898ebd45dabd23dee0a9e16e5d (127.5.9.129:38013), f2b8dbbab98f4b2f9aff6ff6f2f855da (127.5.9.130:34701)
I20260809 02:09:44.029335 6861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:44.026342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:52277 (local address 127.5.9.129:38013)
0809 02:09:44.026386 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:44.026388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:44.026579 (+ 191us) server_negotiation.cc:407] Connection header received
0809 02:09:44.026645 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:44.026647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:44.026679 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:44.026751 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:44.026940 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.027477 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.028525 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.028962 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.029035 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:44.029047 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:44.029213 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 02:09:44.029255 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:09:44.029356 6891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:44.026136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:38013 (local address 127.5.9.131:52277)
0809 02:09:44.026499 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:44.026508 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:44.026585 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:44.026754 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:44.026755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:44.026757 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:44.026847 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:44.026849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.027569 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.027570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:44.028423 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:44.028426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.029049 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.029050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:44.029130 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:44.029135 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:09:44.029196 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 02:09:44.029239 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":215,"thread_start_us":92,"threads_started":1}
I20260809 02:09:44.029686 6497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d55790897514affa0ba150d6ff65aec" candidate_uuid: "00bacf201b694fea8e34e9e7d18000cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46757d898ebd45dabd23dee0a9e16e5d" is_pre_election: true
I20260809 02:09:44.029835 6497 raft_consensus.cc:2454] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00bacf201b694fea8e34e9e7d18000cc in term 0.
I20260809 02:09:44.029929 6859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:44.026262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:39023 (local address 127.5.9.130:34701)
0809 02:09:44.026334 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:44.026336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:44.026346 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:09:44.026376 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:44.026377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:44.026386 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:44.026439 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:44.026649 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.027441 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.028862 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.029369 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.029435 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:44.029446 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:44.029818 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 02:09:44.029861 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 02:09:44.029953 6728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:44.026136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:34701 (local address 127.5.9.131:39023)
0809 02:09:44.026202 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:44.026214 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:44.026266 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:44.026478 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:44.026479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:44.026481 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:44.026559 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:44.026561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.027545 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.027546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:44.028767 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:44.028770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.029637 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.029639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:44.029725 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:44.029729 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:09:44.029812 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 02:09:44.029855 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":63}
I20260809 02:09:44.030035 6722 leader_election.cc:302] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00bacf201b694fea8e34e9e7d18000cc, 46757d898ebd45dabd23dee0a9e16e5d; no voters:
I20260809 02:09:44.030165 6870 raft_consensus.cc:2781] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:09:44.030193 6870 raft_consensus.cc:482] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:09:44.030215 6870 raft_consensus.cc:3037] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:44.030184 6646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d55790897514affa0ba150d6ff65aec" candidate_uuid: "00bacf201b694fea8e34e9e7d18000cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" is_pre_election: true
I20260809 02:09:44.030282 6646 raft_consensus.cc:2454] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00bacf201b694fea8e34e9e7d18000cc in term 0.
I20260809 02:09:44.031239 6870 raft_consensus.cc:504] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.031397 6870 leader_election.cc:288] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [CANDIDATE]: Term 1 election: Requested vote from peers 46757d898ebd45dabd23dee0a9e16e5d (127.5.9.129:38013), f2b8dbbab98f4b2f9aff6ff6f2f855da (127.5.9.130:34701)
I20260809 02:09:44.031595 6497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d55790897514affa0ba150d6ff65aec" candidate_uuid: "00bacf201b694fea8e34e9e7d18000cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46757d898ebd45dabd23dee0a9e16e5d"
I20260809 02:09:44.031622 6646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d55790897514affa0ba150d6ff65aec" candidate_uuid: "00bacf201b694fea8e34e9e7d18000cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da"
I20260809 02:09:44.031685 6646 raft_consensus.cc:3037] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:44.031684 6497 raft_consensus.cc:3037] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:44.032519 6646 raft_consensus.cc:2454] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00bacf201b694fea8e34e9e7d18000cc in term 1.
I20260809 02:09:44.032588 6497 raft_consensus.cc:2454] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00bacf201b694fea8e34e9e7d18000cc in term 1.
I20260809 02:09:44.032749 6720 leader_election.cc:302] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00bacf201b694fea8e34e9e7d18000cc, f2b8dbbab98f4b2f9aff6ff6f2f855da; no voters:
I20260809 02:09:44.032840 6870 raft_consensus.cc:2781] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:44.032963 6870 raft_consensus.cc:686] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [term 1 LEADER]: Becoming Leader. State: Replica: 00bacf201b694fea8e34e9e7d18000cc, State: Running, Role: LEADER
I20260809 02:09:44.033032 6870 consensus_queue.cc:237] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.033866 6332 catalog_manager.cc:5495] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 00bacf201b694fea8e34e9e7d18000cc (127.5.9.131). New cstate: current_term: 1 leader_uuid: "00bacf201b694fea8e34e9e7d18000cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } health_report { overall_health: HEALTHY } } }
W20260809 02:09:44.045423 6547 tablet.cc:2369] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 02:09:44 dist-test-slave-p3t5 krb5kdc[6286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241383, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:44.066943 6899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:44.058545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:44495 (local address 127.0.0.1:44316)
0809 02:09:44.058733 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:44.058745 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:44.058847 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:44.059000 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:44.059000 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:44.059447 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:44.059501 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:44.059505 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.060196 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.060197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:44.060745 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:44.060746 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.061115 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.061116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:44.061169 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:44.061815 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:44.061820 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:44.063533 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:44.065672 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:44.065688 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:44.065690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:44.065959 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:44.066202 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:44.066202 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:44.066203 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:44.066270 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:44.066575 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:44.066582 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:44.066632 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 02:09:44.066686 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 02:09:44.066860 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":2,"thread_start_us":96,"threads_started":1}
I20260809 02:09:44.067065 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:44.058624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44316 (local address 127.5.9.190:44495)
0809 02:09:44.058704 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:44.058707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:44.058826 (+ 119us) server_negotiation.cc:407] Connection header received
0809 02:09:44.058906 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:44.058907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:44.058918 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:44.058995 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:44.059591 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.060103 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.060831 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.061024 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.061111 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:44.063638 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:44.063640 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:44.063641 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:44.063651 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:44.065515 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:44.066044 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:44.066046 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:44.066046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:44.066118 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:44.066339 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:44.066340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:44.066340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:44.066507 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:44.066582 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:44.066748 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 02:09:44.066932 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":54}
I20260809 02:09:44.069710 6332 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:44316:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 02:09:44.070077 6332 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:09:44.071693 6476 tablet_service.cc:1467] Processing CreateTablet for tablet 35d6b242326341788b603497bec0b052 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6e0591a890994f5b8d4cdbaf5b0c4dc9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:09:44.071768 6623 tablet_service.cc:1467] Processing CreateTablet for tablet 35d6b242326341788b603497bec0b052 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6e0591a890994f5b8d4cdbaf5b0c4dc9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:09:44.071746 6770 tablet_service.cc:1467] Processing CreateTablet for tablet 35d6b242326341788b603497bec0b052 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6e0591a890994f5b8d4cdbaf5b0c4dc9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:09:44.071889 6476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35d6b242326341788b603497bec0b052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:44.071913 6623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35d6b242326341788b603497bec0b052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:44.071913 6770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35d6b242326341788b603497bec0b052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:44.073577 6864 tablet_bootstrap.cc:492] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da: Bootstrap starting.
I20260809 02:09:44.073593 6862 tablet_bootstrap.cc:492] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d: Bootstrap starting.
I20260809 02:09:44.073974 6864 tablet_bootstrap.cc:654] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:44.073980 6862 tablet_bootstrap.cc:654] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:44.074994 6864 tablet_bootstrap.cc:492] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da: No bootstrap required, opened a new log
I20260809 02:09:44.075001 6862 tablet_bootstrap.cc:492] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d: No bootstrap required, opened a new log
I20260809 02:09:44.075033 6864 ts_tablet_manager.cc:1397] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:44.075047 6862 ts_tablet_manager.cc:1397] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:09:44.075163 6864 raft_consensus.cc:348] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.075224 6864 raft_consensus.cc:374] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:44.075239 6864 raft_consensus.cc:729] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b8dbbab98f4b2f9aff6ff6f2f855da, State: Initialized, Role: FOLLOWER
I20260809 02:09:44.075178 6862 raft_consensus.cc:348] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.075270 6864 consensus_queue.cc:260] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.075258 6862 raft_consensus.cc:374] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:44.075299 6862 raft_consensus.cc:729] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46757d898ebd45dabd23dee0a9e16e5d, State: Initialized, Role: FOLLOWER
I20260809 02:09:44.075330 6864 ts_tablet_manager.cc:1428] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:44.075341 6862 consensus_queue.cc:260] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.075441 6862 ts_tablet_manager.cc:1428] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:44.075433 6869 tablet_replica.cc:439] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } }
I20260809 02:09:44.075541 6869 tablet_replica.cc:442] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:09:44.075731 6863 tablet_bootstrap.cc:492] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc: Bootstrap starting.
I20260809 02:09:44.075738 6872 tablet_replica.cc:439] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } }
I20260809 02:09:44.075812 6872 tablet_replica.cc:442] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:09:44.076242 6863 tablet_bootstrap.cc:654] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:44.077040 6863 tablet_bootstrap.cc:492] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc: No bootstrap required, opened a new log
I20260809 02:09:44.077085 6863 ts_tablet_manager.cc:1397] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:44.077260 6863 raft_consensus.cc:348] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.077322 6863 raft_consensus.cc:374] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:44.077334 6863 raft_consensus.cc:729] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00bacf201b694fea8e34e9e7d18000cc, State: Initialized, Role: FOLLOWER
I20260809 02:09:44.077389 6863 consensus_queue.cc:260] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.077492 6863 ts_tablet_manager.cc:1428] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:09:44.077636 6870 tablet_replica.cc:439] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } }
I20260809 02:09:44.077754 6870 tablet_replica.cc:442] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 02:09:44.160077 6839 tablet.cc:2369] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:09:44.205009 6870 raft_consensus.cc:482] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:09:44.205171 6870 raft_consensus.cc:504] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.205422 6870 leader_election.cc:288] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46757d898ebd45dabd23dee0a9e16e5d (127.5.9.129:38013), f2b8dbbab98f4b2f9aff6ff6f2f855da (127.5.9.130:34701)
I20260809 02:09:44.205680 6497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d6b242326341788b603497bec0b052" candidate_uuid: "00bacf201b694fea8e34e9e7d18000cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46757d898ebd45dabd23dee0a9e16e5d" is_pre_election: true
I20260809 02:09:44.205731 6646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d6b242326341788b603497bec0b052" candidate_uuid: "00bacf201b694fea8e34e9e7d18000cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" is_pre_election: true
I20260809 02:09:44.205816 6646 raft_consensus.cc:2454] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00bacf201b694fea8e34e9e7d18000cc in term 0.
I20260809 02:09:44.205816 6497 raft_consensus.cc:2454] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00bacf201b694fea8e34e9e7d18000cc in term 0.
I20260809 02:09:44.205971 6720 leader_election.cc:302] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00bacf201b694fea8e34e9e7d18000cc, f2b8dbbab98f4b2f9aff6ff6f2f855da; no voters:
I20260809 02:09:44.206102 6870 raft_consensus.cc:2781] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:09:44.206163 6870 raft_consensus.cc:482] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:09:44.206178 6870 raft_consensus.cc:3037] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:44.206908 6870 raft_consensus.cc:504] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.207067 6870 leader_election.cc:288] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [CANDIDATE]: Term 1 election: Requested vote from peers 46757d898ebd45dabd23dee0a9e16e5d (127.5.9.129:38013), f2b8dbbab98f4b2f9aff6ff6f2f855da (127.5.9.130:34701)
I20260809 02:09:44.207286 6646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d6b242326341788b603497bec0b052" candidate_uuid: "00bacf201b694fea8e34e9e7d18000cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da"
I20260809 02:09:44.207280 6497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35d6b242326341788b603497bec0b052" candidate_uuid: "00bacf201b694fea8e34e9e7d18000cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46757d898ebd45dabd23dee0a9e16e5d"
I20260809 02:09:44.207391 6497 raft_consensus.cc:3037] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:44.207391 6646 raft_consensus.cc:3037] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:44.208033 6646 raft_consensus.cc:2454] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00bacf201b694fea8e34e9e7d18000cc in term 1.
I20260809 02:09:44.208110 6497 raft_consensus.cc:2454] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00bacf201b694fea8e34e9e7d18000cc in term 1.
I20260809 02:09:44.208282 6720 leader_election.cc:302] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00bacf201b694fea8e34e9e7d18000cc, f2b8dbbab98f4b2f9aff6ff6f2f855da; no voters:
I20260809 02:09:44.208413 6870 raft_consensus.cc:2781] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:44.208501 6870 raft_consensus.cc:686] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [term 1 LEADER]: Becoming Leader. State: Replica: 00bacf201b694fea8e34e9e7d18000cc, State: Running, Role: LEADER
I20260809 02:09:44.208583 6870 consensus_queue.cc:237] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } }
I20260809 02:09:44.208949 6892 tablet_replica.cc:439] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc: TxnStatusTablet state changed. Reason: New leader 00bacf201b694fea8e34e9e7d18000cc. Latest consensus state: current_term: 1 leader_uuid: "00bacf201b694fea8e34e9e7d18000cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } }
I20260809 02:09:44.209044 6892 tablet_replica.cc:442] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:09:44.209237 6905 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 02:09:44.209411 6905 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 02:09:44.209314 6332 catalog_manager.cc:5495] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 00bacf201b694fea8e34e9e7d18000cc (127.5.9.131). New cstate: current_term: 1 leader_uuid: "00bacf201b694fea8e34e9e7d18000cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } health_report { overall_health: HEALTHY } } }
I20260809 02:09:44.225519 6899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:44.222700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:45965 (local address 127.0.0.1:51398)
0809 02:09:44.222781 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:44.222798 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:44.222867 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:44.223075 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:44.223076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:44.223077 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:09:44.223142 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:44.223144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.223834 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.223835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:44.224445 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:44.224447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.224868 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.224869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:44.224945 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:44.224962 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:09:44.225324 (+ 362us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:09:44.225335 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 02:09:44.225383 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 02:09:44.225444 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 02:09:44.225518 6860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:44.222791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51398 (local address 127.5.9.131:45965)
0809 02:09:44.222860 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:44.222863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:44.222873 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:09:44.222968 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:44.222969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:44.222995 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:44.223069 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:09:44.223216 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.223741 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.224546 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.224784 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.224867 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:44.225072 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:09:44.225243 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:09:44.225328 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:44.225385 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 02:09:44.225409 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:09:44.227150 6497 raft_consensus.cc:1261] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d [term 1 FOLLOWER]: Refusing update from remote peer 00bacf201b694fea8e34e9e7d18000cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:09:44.227146 6646 raft_consensus.cc:1261] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 1 FOLLOWER]: Refusing update from remote peer 00bacf201b694fea8e34e9e7d18000cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:09:44.227478 6892 consensus_queue.cc:1035] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:09:44.227809 6892 consensus_queue.cc:1035] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:09:44.228195 6869 tablet_replica.cc:439] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da: TxnStatusTablet state changed. Reason: New leader 00bacf201b694fea8e34e9e7d18000cc. Latest consensus state: current_term: 1 leader_uuid: "00bacf201b694fea8e34e9e7d18000cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } }
I20260809 02:09:44.228308 6869 tablet_replica.cc:442] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:09:44.228569 6872 tablet_replica.cc:439] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d: TxnStatusTablet state changed. Reason: New leader 00bacf201b694fea8e34e9e7d18000cc. Latest consensus state: current_term: 1 leader_uuid: "00bacf201b694fea8e34e9e7d18000cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } }
I20260809 02:09:44.228672 6872 tablet_replica.cc:442] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:09:44.229378 6870 tablet_replica.cc:439] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00bacf201b694fea8e34e9e7d18000cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } }
I20260809 02:09:44.229472 6870 tablet_replica.cc:442] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:09:44.229506 6869 tablet_replica.cc:439] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00bacf201b694fea8e34e9e7d18000cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } }
I20260809 02:09:44.229586 6869 tablet_replica.cc:442] T 35d6b242326341788b603497bec0b052 P f2b8dbbab98f4b2f9aff6ff6f2f855da: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:09:44.230602 6892 tablet_replica.cc:439] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00bacf201b694fea8e34e9e7d18000cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } }
I20260809 02:09:44.230710 6892 tablet_replica.cc:442] T 35d6b242326341788b603497bec0b052 P 00bacf201b694fea8e34e9e7d18000cc: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:09:44.231328 6872 tablet_replica.cc:439] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00bacf201b694fea8e34e9e7d18000cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 } } peers { permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 } } peers { permanent_uuid: "00bacf201b694fea8e34e9e7d18000cc" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45965 } } }
I20260809 02:09:44.231421 6872 tablet_replica.cc:442] T 35d6b242326341788b603497bec0b052 P 46757d898ebd45dabd23dee0a9e16e5d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:09:44.234900 6860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:44.232048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51412 (local address 127.5.9.131:45965)
0809 02:09:44.232088 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:44.232089 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:44.232100 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:09:44.232181 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:44.232182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:44.232191 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:44.232228 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:09:44.232358 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.232843 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.234059 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.234372 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.234436 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:44.234623 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:09:44.234657 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:09:44.234715 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:44.234786 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 02:09:44.234806 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 02:09:44.240706 6845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:44.237880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:45965 (local address 127.5.9.131:51835)
0809 02:09:44.237986 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:44.237996 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:44.238043 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:44.238163 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:44.238163 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:44.238164 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:09:44.238613 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:44.238615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.239264 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.239265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:44.239867 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:44.239868 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.240252 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.240253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:44.240315 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:44.240343 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:09:44.240538 (+ 195us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:09:44.240540 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:09:44.240593 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 02:09:44.240636 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":1}
I20260809 02:09:44.240706 6860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:44.237933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:51835 (local address 127.5.9.131:45965)
0809 02:09:44.237969 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:44.237971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:44.238027 (+ 56us) server_negotiation.cc:407] Connection header received
0809 02:09:44.238098 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:44.238099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:44.238108 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:44.238160 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:09:44.238701 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.239175 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.239980 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.240173 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.240253 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:44.240438 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:09:44.240477 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:09:44.240539 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:44.240598 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 02:09:44.240615 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":15}
I20260809 02:09:44.243772 6646 raft_consensus.cc:1261] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 1 FOLLOWER]: Refusing update from remote peer 00bacf201b694fea8e34e9e7d18000cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:09:44.243956 6497 raft_consensus.cc:1261] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d [term 1 FOLLOWER]: Refusing update from remote peer 00bacf201b694fea8e34e9e7d18000cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:09:44.244067 6910 consensus_queue.cc:1035] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b8dbbab98f4b2f9aff6ff6f2f855da" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 34701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:09:44.244189 6892 consensus_queue.cc:1035] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "46757d898ebd45dabd23dee0a9e16e5d" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 38013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:09:44.245540 6881 mvcc.cc:205] Tried to move back new op lower bound from 7316444709860749312 to 7316444709000060928. Current Snapshot: MvccSnapshot[applied={T|T < 7316444709860749312}]
I20260809 02:09:44.257565 6331 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:09:44.257726 6331 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:09:44.258404 6331 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e105c6c48a449d687d847d63677307f: Sending DeleteTablet for 3 replicas of tablet 0d55790897514affa0ba150d6ff65aec
I20260809 02:09:44.258733 6476 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d55790897514affa0ba150d6ff65aec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:44 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:47626
I20260809 02:09:44.258802 6770 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d55790897514affa0ba150d6ff65aec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:44 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:51394
I20260809 02:09:44.258790 6623 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d55790897514affa0ba150d6ff65aec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:44 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:48838
I20260809 02:09:44.258845 6887 tablet_replica.cc:330] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d: stopping tablet replica
I20260809 02:09:44.258890 6885 tablet_replica.cc:330] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da: stopping tablet replica
I20260809 02:09:44.258893 6887 raft_consensus.cc:2229] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:09:44.258939 6887 raft_consensus.cc:2258] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:44.258944 6885 raft_consensus.cc:2229] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:09:44.258980 6885 raft_consensus.cc:2258] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:44.259042 6886 tablet_replica.cc:330] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc: stopping tablet replica
I20260809 02:09:44.259097 6886 raft_consensus.cc:2229] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:09:44.259203 6886 raft_consensus.cc:2258] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:44.259299 6887 ts_tablet_manager.cc:1905] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:44.259670 6885 ts_tablet_manager.cc:1905] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:44.260978 6886 ts_tablet_manager.cc:1905] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:44.261188 6885 ts_tablet_manager.cc:1918] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:09:44.261256 6885 log.cc:1196] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/0d55790897514affa0ba150d6ff65aec
I20260809 02:09:44.261255 6887 ts_tablet_manager.cc:1918] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:09:44.261325 6887 log.cc:1196] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-0/wal/wals/0d55790897514affa0ba150d6ff65aec
I20260809 02:09:44.261490 6885 ts_tablet_manager.cc:1939] T 0d55790897514affa0ba150d6ff65aec P f2b8dbbab98f4b2f9aff6ff6f2f855da: Deleting consensus metadata
I20260809 02:09:44.261564 6887 ts_tablet_manager.cc:1939] T 0d55790897514affa0ba150d6ff65aec P 46757d898ebd45dabd23dee0a9e16e5d: Deleting consensus metadata
I20260809 02:09:44.261881 6316 catalog_manager.cc:4841] TS f2b8dbbab98f4b2f9aff6ff6f2f855da (127.5.9.130:34701): tablet 0d55790897514affa0ba150d6ff65aec (table test-table [id=5d84f226cc754718b652a8ff98baeaf3]) successfully deleted
I20260809 02:09:44.262599 6886 ts_tablet_manager.cc:1918] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:09:44.262679 6886 log.cc:1196] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/0d55790897514affa0ba150d6ff65aec
I20260809 02:09:44.262914 6886 ts_tablet_manager.cc:1939] T 0d55790897514affa0ba150d6ff65aec P 00bacf201b694fea8e34e9e7d18000cc: Deleting consensus metadata
I20260809 02:09:44.263283 6317 catalog_manager.cc:4841] TS 00bacf201b694fea8e34e9e7d18000cc (127.5.9.131:45965): tablet 0d55790897514affa0ba150d6ff65aec (table test-table [id=5d84f226cc754718b652a8ff98baeaf3]) successfully deleted
I20260809 02:09:44.263309 6318 catalog_manager.cc:4841] TS 46757d898ebd45dabd23dee0a9e16e5d (127.5.9.129:38013): tablet 0d55790897514affa0ba150d6ff65aec (table test-table [id=5d84f226cc754718b652a8ff98baeaf3]) successfully deleted
Aug 09 02:09:44 dist-test-slave-p3t5 krb5kdc[6286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241383, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.9.129@KRBTEST.COM
I20260809 02:09:44.274361 6861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:44.264423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47648 (local address 127.5.9.129:38013)
0809 02:09:44.264475 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:44.264477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:44.264557 (+ 80us) server_negotiation.cc:407] Connection header received
0809 02:09:44.264637 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:44.264638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:44.264646 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:44.264698 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:44.265418 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.265983 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.267093 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.267302 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.267360 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:44.271087 (+ 3727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:44.271102 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:44.271102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:44.271113 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:44.272961 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:44.273378 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:44.273382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:44.273383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:44.273453 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:44.273699 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:44.273700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:44.273700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:44.273876 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:44.274049 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:44.274133 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 02:09:44.274231 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260809 02:09:44.274725 6539 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47648
I20260809 02:09:44.282301 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:44.275942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44318 (local address 127.5.9.190:44495)
0809 02:09:44.276017 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:44.276020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:44.276027 (+ 7us) server_negotiation.cc:407] Connection header received
0809 02:09:44.276054 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:44.276055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:44.276071 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:44.276134 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:44.276685 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.277208 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.278036 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:44.278261 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:44.278342 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:44.279627 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:44.279629 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:44.279629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:44.279638 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:44.281017 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:44.281400 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:44.281401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:44.281402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:44.281473 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:44.281722 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:44.281723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:44.281724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:44.281862 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:44.281985 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:44.282084 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 02:09:44.282176 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 02:09:44.282583 6331 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44318
I20260809 02:09:44.283291 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 6409
I20260809 02:09:44.294764 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 6555
I20260809 02:09:44.303691 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 6704
I20260809 02:09:44.312043 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 6299
2026-08-09T02:09:44Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2350 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:09:44 dist-test-slave-p3t5 krb5kdc[6923](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:09:44 dist-test-slave-p3t5 krb5kdc[6923](info): set up 2 sockets
Aug 09 02:09:44 dist-test-slave-p3t5 krb5kdc[6923](info): commencing operation
krb5kdc: starting...
W20260809 02:09:45.393757 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:09:45 dist-test-slave-p3t5 krb5kdc[6923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:09:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:09:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:09:45.528373 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:35481
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:33423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:35481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:45.615388 6936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:45.615630 6936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:45.615655 6936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:45.616806 6936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:09:45.616844 6936 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 02:09:45.616858 6936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:45.616871 6936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:45.616884 6936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:09:45.616896 6936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:09:45.618398 6936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:33423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:35481
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:35481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:45.618615 6936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:45.618813 6936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:09:45.624153 6936 server_base.cc:1034] running on GCE node
W20260809 02:09:45.624195 6942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:45.624115 6941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:45.624122 6944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:45.624624 6936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:45.624982 6936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:45.626209 6936 hybrid_clock.cc:648] HybridClock initialized: now 1786241385626184 us; error 41 us; skew 500 ppm
Aug 09 02:09:45 dist-test-slave-p3t5 krb5kdc[6923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241385, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:45.628961 6936 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:09:45.629482 6936 webserver.cc:460] Webserver started at http://127.5.9.190:46211/ using document root <none> and password file <none>
I20260809 02:09:45.629772 6936 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:45.629849 6936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:45.630025 6936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:45.631145 6936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "47e744c6318b42c6826cd84617ec0e47"
format_stamp: "Formatted at 2026-08-09 02:09:45 on dist-test-slave-p3t5"
server_key: "6d6128109d24b5812a07f8e59d7f8ec1"
server_key_iv: "eaf4804ef8865416bb9a780480a2df8e"
server_key_version: "encryptionkey@0"
I20260809 02:09:45.631536 6936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "47e744c6318b42c6826cd84617ec0e47"
format_stamp: "Formatted at 2026-08-09 02:09:45 on dist-test-slave-p3t5"
server_key: "6d6128109d24b5812a07f8e59d7f8ec1"
server_key_iv: "eaf4804ef8865416bb9a780480a2df8e"
server_key_version: "encryptionkey@0"
I20260809 02:09:45.632946 6936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:09:45.633703 6951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:45.633909 6936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:45.634020 6936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "47e744c6318b42c6826cd84617ec0e47"
format_stamp: "Formatted at 2026-08-09 02:09:45 on dist-test-slave-p3t5"
server_key: "6d6128109d24b5812a07f8e59d7f8ec1"
server_key_iv: "eaf4804ef8865416bb9a780480a2df8e"
server_key_version: "encryptionkey@0"
I20260809 02:09:45.634106 6936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:45.650496 6936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:45.653074 6936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:45.653208 6936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:45.658694 6936 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:35481
I20260809 02:09:45.658743 7015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:35481 every 8 connection(s)
I20260809 02:09:45.659127 6936 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:09:45.659813 7016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:45.662463 7016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47: Bootstrap starting.
I20260809 02:09:45.663143 7016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:45.663477 7016 log.cc:824] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:45.664301 7016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47: No bootstrap required, opened a new log
I20260809 02:09:45.664824 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 6936
I20260809 02:09:45.664943 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:09:45.665138 5158 external_mini_cluster.cc:1428] Setting key 474b023ab70e9fab002dd2cfb755a4eb
I20260809 02:09:45.666399 7016 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e744c6318b42c6826cd84617ec0e47" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 35481 } }
I20260809 02:09:45.666580 7016 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:45.666630 7016 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47e744c6318b42c6826cd84617ec0e47, State: Initialized, Role: FOLLOWER
I20260809 02:09:45.666798 7016 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e744c6318b42c6826cd84617ec0e47" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 35481 } }
I20260809 02:09:45.666911 7016 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:09:45.666954 7016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:09:45.667026 7016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:45.667692 7016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e744c6318b42c6826cd84617ec0e47" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 35481 } }
I20260809 02:09:45.667826 7016 leader_election.cc:302] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47e744c6318b42c6826cd84617ec0e47; no voters:
I20260809 02:09:45.668052 7016 leader_election.cc:288] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:09:45.668084 7021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:45.668202 7021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [term 1 LEADER]: Becoming Leader. State: Replica: 47e744c6318b42c6826cd84617ec0e47, State: Running, Role: LEADER
I20260809 02:09:45.668350 7016 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:09:45.668305 7021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e744c6318b42c6826cd84617ec0e47" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 35481 } }
I20260809 02:09:45.668722 7023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47e744c6318b42c6826cd84617ec0e47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e744c6318b42c6826cd84617ec0e47" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 35481 } } }
I20260809 02:09:45.668741 7022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47e744c6318b42c6826cd84617ec0e47. Latest consensus state: current_term: 1 leader_uuid: "47e744c6318b42c6826cd84617ec0e47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47e744c6318b42c6826cd84617ec0e47" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 35481 } } }
I20260809 02:09:45.668828 7023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [sys.catalog]: This master's current role is: LEADER
I20260809 02:09:45.668852 7022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47 [sys.catalog]: This master's current role is: LEADER
I20260809 02:09:45.669817 7027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:09:45.670429 7027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:09:45.672292 7027 catalog_manager.cc:1347] Generated new cluster ID: 779ae0d5aa6b4cca8d121cc249576e97
I20260809 02:09:45.672365 7027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:09:45 dist-test-slave-p3t5 krb5kdc[6923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:45.676990 7020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:45.665747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46550 (local address 127.5.9.190:35481)
0809 02:09:45.665951 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:45.665959 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:45.665987 (+ 28us) server_negotiation.cc:407] Connection header received
0809 02:09:45.666425 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:45.666432 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:45.666483 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:45.666639 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:45.667425 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:45.668675 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:45.669862 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:45.670423 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:45.670574 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:45.673336 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:45.673340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:45.673340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:45.673353 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:45.675222 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:45.675652 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:45.675655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:45.675656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:45.675789 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:45.676037 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:45.676038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:45.676039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:45.676317 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:45.676531 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:45.676607 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 02:09:45.676769 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":80,"threads_started":1}
I20260809 02:09:45.730404 7027 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:09:45.731050 7027 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:09:45.737654 7027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47: Generated new TSK 0
I20260809 02:09:45.737941 7027 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:09:45.777278 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:35481
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:33423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:45.887169 7042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:45.887418 7042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:45.887445 7042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:45.888593 7042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:45.888635 7042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:45.888656 7042 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 02:09:45.888669 7042 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:09:45.888685 7042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:09:45.890197 7042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:33423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.9.190:35481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:45.890431 7042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:45.890628 7042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:09:45.896016 7042 server_base.cc:1034] running on GCE node
W20260809 02:09:45.895982 7048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:45.895982 7047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:45.895977 7050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:45.896384 7042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:45.896669 7042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:45.897825 7042 hybrid_clock.cc:648] HybridClock initialized: now 1786241385897819 us; error 64 us; skew 500 ppm
Aug 09 02:09:45 dist-test-slave-p3t5 krb5kdc[6923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241385, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:45.900568 7042 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:09:45.901111 7042 webserver.cc:460] Webserver started at http://127.5.9.129:46699/ using document root <none> and password file <none>
I20260809 02:09:45.901386 7042 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:45.901437 7042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:45.901669 7042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:45.902817 7042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "30ff7069fc3f4a0fa8a472de7ba60094"
format_stamp: "Formatted at 2026-08-09 02:09:45 on dist-test-slave-p3t5"
server_key: "8edcbc6656919a9205cc83ffa255d479"
server_key_iv: "95125b4f1a1748abef93768fdac4cd06"
server_key_version: "encryptionkey@0"
I20260809 02:09:45.903216 7042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "30ff7069fc3f4a0fa8a472de7ba60094"
format_stamp: "Formatted at 2026-08-09 02:09:45 on dist-test-slave-p3t5"
server_key: "8edcbc6656919a9205cc83ffa255d479"
server_key_iv: "95125b4f1a1748abef93768fdac4cd06"
server_key_version: "encryptionkey@0"
I20260809 02:09:45.904784 7042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:09:45.905601 7057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:45.905859 7042 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:09:45.905972 7042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "30ff7069fc3f4a0fa8a472de7ba60094"
format_stamp: "Formatted at 2026-08-09 02:09:45 on dist-test-slave-p3t5"
server_key: "8edcbc6656919a9205cc83ffa255d479"
server_key_iv: "95125b4f1a1748abef93768fdac4cd06"
server_key_version: "encryptionkey@0"
I20260809 02:09:45.906064 7042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:45.918908 7042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:45.922253 7042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:45.922454 7042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:45.923170 7042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:45.923229 7042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:45.923266 7042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:45.923285 7042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:09:45 dist-test-slave-p3t5 krb5kdc[6923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241385, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:45.931192 7042 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:36689
I20260809 02:09:45.931296 7181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:36689 every 8 connection(s)
I20260809 02:09:45.931658 7042 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:09:45.934417 7020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:45.924092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:35019 (local address 127.5.9.190:35481)
0809 02:09:45.924153 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:45.924155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:45.924404 (+ 249us) server_negotiation.cc:407] Connection header received
0809 02:09:45.925057 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:45.925059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:45.925071 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:45.925130 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:45.926112 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:45.927019 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:45.928191 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:45.928433 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:45.928514 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:45.931270 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:45.931272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:45.931272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:45.931284 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:45.932993 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:45.933412 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:45.933413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:45.933413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:45.933472 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:45.933789 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:45.933790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:45.933790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:45.933933 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:45.933990 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:45.934267 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 02:09:45.934348 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 02:09:45.934581 7066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:45.924072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:35481 (local address 127.5.9.129:35019)
0809 02:09:45.924257 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:45.924302 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:45.924928 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:45.925267 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:45.925272 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:45.925693 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:45.925989 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:45.925996 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:45.927172 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:45.927172 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:45.928082 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:45.928084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:45.928559 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:45.928560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:45.928702 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:45.929318 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:45.929334 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:45.931133 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:45.933088 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:45.933096 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:45.933098 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:45.933337 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:45.933533 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:45.933533 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:45.933534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:45.933720 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:45.933996 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:45.934003 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:45.934134 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 02:09:45.934216 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 02:09:45.934352 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":109,"threads_started":1}
I20260809 02:09:45.935163 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 7042
I20260809 02:09:45.935267 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:09:45.935460 5158 external_mini_cluster.cc:1428] Setting key a4f6964c7cbbb0b82fe6a9d5887ffe53
I20260809 02:09:45.935822 7182 heartbeater.cc:347] Connected to a master server at 127.5.9.190:35481
I20260809 02:09:45.935941 7182 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:45.936151 7182 heartbeater.cc:511] Master 127.5.9.190:35481 requested a full tablet report, sending...
I20260809 02:09:45.936736 6969 ts_manager.cc:194] Registered new tserver with Master: 30ff7069fc3f4a0fa8a472de7ba60094 (127.5.9.129:36689)
I20260809 02:09:45.937685 6969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:35019
I20260809 02:09:45.943298 7020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:45.937149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:36623 (local address 127.5.9.190:35481)
0809 02:09:45.937191 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:45.937193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:45.937338 (+ 145us) server_negotiation.cc:407] Connection header received
0809 02:09:45.937386 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:45.937387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:45.937399 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:45.937450 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:45.937877 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:45.938564 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:45.939354 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:45.939572 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:45.939651 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:45.940668 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:45.940669 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:45.940669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:45.940676 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:45.942106 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:45.942468 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:45.942469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:45.942470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:45.942530 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:45.942747 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:45.942748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:45.942749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:45.942868 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:45.942942 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:45.943107 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 02:09:45.943195 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:09:45.943312 7188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:45.937082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:35481 (local address 127.5.9.129:36623)
0809 02:09:45.937254 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:45.937265 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:45.937328 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:45.937451 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:45.937452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:45.937694 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:45.937764 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:45.937767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:45.938669 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:45.938670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:45.939261 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:45.939263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:45.939650 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:45.939651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:45.939732 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:45.940239 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:45.940243 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:45.940563 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:45.942199 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:45.942201 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:45.942201 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:45.942391 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:45.942599 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:45.942600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:45.942601 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:45.942666 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:45.942947 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:45.942948 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:45.942994 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 02:09:45.943081 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 02:09:45.943196 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:09:45.965356 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:35481
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:33423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:46.056509 7191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:46.056761 7191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:46.056795 7191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:46.057999 7191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:46.058046 7191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:46.058068 7191 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 02:09:46.058081 7191 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:09:46.058099 7191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:09:46.059582 7191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:33423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.9.190:35481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:46.059795 7191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:46.059988 7191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:46.065363 7196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:46.065337 7197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:46.065485 7191 server_base.cc:1034] running on GCE node
W20260809 02:09:46.065354 7199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:46.065852 7191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:46.066138 7191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:46.067322 7191 hybrid_clock.cc:648] HybridClock initialized: now 1786241386067286 us; error 51 us; skew 500 ppm
Aug 09 02:09:46 dist-test-slave-p3t5 krb5kdc[6923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:46.070468 7191 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:09:46.070960 7191 webserver.cc:460] Webserver started at http://127.5.9.130:44057/ using document root <none> and password file <none>
I20260809 02:09:46.071208 7191 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:46.071287 7191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:46.071450 7191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:46.072600 7191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "821f1f626975452a92167f161383dbee"
format_stamp: "Formatted at 2026-08-09 02:09:46 on dist-test-slave-p3t5"
server_key: "9560c3cb02de70b174e8186875d0991c"
server_key_iv: "2d48c3bcbf184e918393799270e59d97"
server_key_version: "encryptionkey@0"
I20260809 02:09:46.073002 7191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "821f1f626975452a92167f161383dbee"
format_stamp: "Formatted at 2026-08-09 02:09:46 on dist-test-slave-p3t5"
server_key: "9560c3cb02de70b174e8186875d0991c"
server_key_iv: "2d48c3bcbf184e918393799270e59d97"
server_key_version: "encryptionkey@0"
I20260809 02:09:46.074558 7191 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:09:46.075430 7207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:46.075652 7191 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:09:46.075780 7191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "821f1f626975452a92167f161383dbee"
format_stamp: "Formatted at 2026-08-09 02:09:46 on dist-test-slave-p3t5"
server_key: "9560c3cb02de70b174e8186875d0991c"
server_key_iv: "2d48c3bcbf184e918393799270e59d97"
server_key_version: "encryptionkey@0"
I20260809 02:09:46.075866 7191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:46.092617 7191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:46.094940 7191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:46.095139 7191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:46.095718 7191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:46.095825 7191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:46.095862 7191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:46.095893 7191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:09:46 dist-test-slave-p3t5 krb5kdc[6923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:46.103430 7191 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:36783
I20260809 02:09:46.103557 7327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:36783 every 8 connection(s)
I20260809 02:09:46.103957 7191 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:09:46.106850 7020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.096286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:50857 (local address 127.5.9.190:35481)
0809 02:09:46.096362 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:46.096364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:46.096642 (+ 278us) server_negotiation.cc:407] Connection header received
0809 02:09:46.097422 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:46.097423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:46.097432 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:46.097474 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:46.098490 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.099064 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.100756 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.101041 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.101123 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.103867 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:46.103869 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:46.103869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:46.103878 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:46.105438 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:46.105926 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:46.105927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:46.105927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:46.105995 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:46.106240 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:46.106252 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:46.106252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:46.106371 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:46.106424 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:46.106676 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 02:09:46.106782 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:09:46.106982 7216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.096258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:35481 (local address 127.5.9.130:50857)
0809 02:09:46.096484 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:46.096537 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:46.097290 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:46.097572 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:46.097577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:46.098030 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:46.098372 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.098380 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.099143 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.099144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.100652 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.100657 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.101153 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.101154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.101310 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.101952 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:46.101961 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:46.103773 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:46.105515 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:46.105522 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:46.105523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:46.105837 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:46.106069 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:46.106069 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:46.106069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:46.106175 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:46.106427 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:46.106431 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:46.106551 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 02:09:46.106629 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 02:09:46.106771 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":109,"threads_started":1}
I20260809 02:09:46.107441 7328 heartbeater.cc:347] Connected to a master server at 127.5.9.190:35481
I20260809 02:09:46.107532 7328 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:46.107740 7328 heartbeater.cc:511] Master 127.5.9.190:35481 requested a full tablet report, sending...
I20260809 02:09:46.108088 6969 ts_manager.cc:194] Registered new tserver with Master: 821f1f626975452a92167f161383dbee (127.5.9.130:36783)
I20260809 02:09:46.108574 6969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:50857
I20260809 02:09:46.112531 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 7191
I20260809 02:09:46.112649 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:09:46.112845 5158 external_mini_cluster.cc:1428] Setting key bf4ae9e128f45a9b5ec232425ffab336
I20260809 02:09:46.114583 7335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.108523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:35481 (local address 127.5.9.130:53369)
0809 02:09:46.108705 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:46.108716 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:46.108777 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:46.108964 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:46.108965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:46.109142 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:46.109198 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.109200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.110028 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.110029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.110539 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.110541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.110899 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.110900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.110956 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.111405 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:46.111410 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:46.111694 (+ 284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:46.113408 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:46.113408 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:46.113409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:46.113700 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:46.113930 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:46.113930 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:46.113930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:46.113980 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:46.114318 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:46.114319 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:46.114357 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 02:09:46.114416 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 02:09:46.114500 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":118,"threads_started":1}
I20260809 02:09:46.114692 7020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.108609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:53369 (local address 127.5.9.190:35481)
0809 02:09:46.108654 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:46.108656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:46.108763 (+ 107us) server_negotiation.cc:407] Connection header received
0809 02:09:46.108855 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:46.108856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:46.108865 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:46.108920 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:46.109286 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.109900 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.110612 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.110827 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.110884 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.111806 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:46.111807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:46.111807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:46.111813 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:46.113316 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:46.113786 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:46.113787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:46.113788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:46.113858 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:46.114037 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:46.114037 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:46.114038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:46.114248 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:46.114325 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:46.114460 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 02:09:46.114549 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:09:46.142906 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:35481
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:33423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:46.232465 7338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:46.232692 7338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:46.232719 7338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:46.234026 7338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:46.234081 7338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:46.234104 7338 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 02:09:46.234145 7338 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:09:46.234171 7338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:09:46.236054 7338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:33423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.9.190:35481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:46.236369 7338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:46.236653 7338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:46.242394 7344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:46.242489 7338 server_base.cc:1034] running on GCE node
W20260809 02:09:46.242358 7343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:46.242384 7346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:46.242815 7338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:46.243101 7338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:46.244299 7338 hybrid_clock.cc:648] HybridClock initialized: now 1786241386244288 us; error 35 us; skew 500 ppm
Aug 09 02:09:46 dist-test-slave-p3t5 krb5kdc[6923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:46.247105 7338 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:09:46.247584 7338 webserver.cc:460] Webserver started at http://127.5.9.131:40487/ using document root <none> and password file <none>
I20260809 02:09:46.247857 7338 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:46.247937 7338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:46.248096 7338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:46.249253 7338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "a33c8858a824442f9fd3fc9805cc318d"
format_stamp: "Formatted at 2026-08-09 02:09:46 on dist-test-slave-p3t5"
server_key: "a36fbca564aee8f3fb8544c95d2d0d76"
server_key_iv: "312562bafeda1491cff994960b388d34"
server_key_version: "encryptionkey@0"
I20260809 02:09:46.249713 7338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "a33c8858a824442f9fd3fc9805cc318d"
format_stamp: "Formatted at 2026-08-09 02:09:46 on dist-test-slave-p3t5"
server_key: "a36fbca564aee8f3fb8544c95d2d0d76"
server_key_iv: "312562bafeda1491cff994960b388d34"
server_key_version: "encryptionkey@0"
I20260809 02:09:46.251204 7338 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:46.252092 7354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:46.252295 7338 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:46.252404 7338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "a33c8858a824442f9fd3fc9805cc318d"
format_stamp: "Formatted at 2026-08-09 02:09:46 on dist-test-slave-p3t5"
server_key: "a36fbca564aee8f3fb8544c95d2d0d76"
server_key_iv: "312562bafeda1491cff994960b388d34"
server_key_version: "encryptionkey@0"
I20260809 02:09:46.252496 7338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:46.275208 7338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:46.277468 7338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:46.277680 7338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:46.278239 7338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:46.278312 7338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:46.278381 7338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:46.278411 7338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:09:46 dist-test-slave-p3t5 krb5kdc[6923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:46.286151 7338 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:39633
I20260809 02:09:46.286621 7338 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:09:46.286230 7474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:39633 every 8 connection(s)
I20260809 02:09:46.288676 7020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.278859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:47369 (local address 127.5.9.190:35481)
0809 02:09:46.278943 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:46.278946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:46.279210 (+ 264us) server_negotiation.cc:407] Connection header received
0809 02:09:46.279817 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:46.279819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:46.279835 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:46.279924 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:46.280943 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.281858 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.282944 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.283147 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.283234 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.285605 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:46.285607 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:46.285608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:46.285637 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:46.287414 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:46.287819 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:46.287820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:46.287820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:46.287878 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:46.288105 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:46.288106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:46.288106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:46.288219 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:46.288274 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:46.288538 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 02:09:46.288609 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:09:46.288921 7363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.278779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:35481 (local address 127.5.9.131:47369)
0809 02:09:46.279075 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:46.279123 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:46.279675 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:46.280044 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:46.280049 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:46.280450 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:46.280788 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.280804 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.281966 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.281967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.282839 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.282842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.283254 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.283256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.283399 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.283998 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:46.284007 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:46.285495 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:46.287489 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:46.287493 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:46.287495 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:46.287728 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:46.287941 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:46.287941 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:46.287942 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:46.288047 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:46.288276 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:46.288281 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:46.288412 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 02:09:46.288503 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 02:09:46.288652 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":120,"threads_started":1}
I20260809 02:09:46.289225 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 7338
I20260809 02:09:46.289314 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:09:46.289412 7475 heartbeater.cc:347] Connected to a master server at 127.5.9.190:35481
I20260809 02:09:46.289527 7475 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:46.289529 5158 external_mini_cluster.cc:1428] Setting key 8945968f4e84c2d9d1af6ee37707275c
I20260809 02:09:46.289754 7475 heartbeater.cc:511] Master 127.5.9.190:35481 requested a full tablet report, sending...
I20260809 02:09:46.290202 6969 ts_manager.cc:194] Registered new tserver with Master: a33c8858a824442f9fd3fc9805cc318d (127.5.9.131:39633)
I20260809 02:09:46.290705 6969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:47369
I20260809 02:09:46.290776 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:09:46.297741 7481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.290645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:35481 (local address 127.5.9.131:55271)
0809 02:09:46.290788 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:46.290807 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:46.290891 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:46.291041 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:46.291042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:46.291217 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:46.291273 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.291274 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.292152 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.292153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.292731 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.292733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.293221 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.293222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.293316 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.294286 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:46.294292 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:46.294807 (+ 515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:46.296517 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:46.296518 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:46.296519 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:46.296764 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:46.297037 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:46.297038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:46.297038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:46.297100 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:46.297419 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:46.297419 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:46.297459 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 02:09:46.297506 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 02:09:46.297592 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260809 02:09:46.297830 7020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.290636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:55271 (local address 127.5.9.190:35481)
0809 02:09:46.290700 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:46.290702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:46.290865 (+ 163us) server_negotiation.cc:407] Connection header received
0809 02:09:46.290964 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:46.290965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:46.290976 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:46.291039 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:46.291348 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.292045 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.292810 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.293125 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.293207 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.294900 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:46.294901 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:46.294902 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:46.294908 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:46.296428 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:46.296867 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:46.296868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:46.296868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:46.296939 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:46.297168 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:46.297169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:46.297170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:46.297347 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:46.297446 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:46.297548 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 02:09:46.297681 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 02:09:46 dist-test-slave-p3t5 krb5kdc[6923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:09:46 dist-test-slave-p3t5 krb5kdc[6923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:46.317967 7020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.310400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46554 (local address 127.5.9.190:35481)
0809 02:09:46.310488 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:46.310490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:46.310517 (+ 27us) server_negotiation.cc:407] Connection header received
0809 02:09:46.310634 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:46.310635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:46.310645 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:46.310705 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:46.311388 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.311904 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.312657 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.312869 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.312948 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.315459 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:46.315461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:46.315461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:46.315469 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:46.316840 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:46.317188 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:46.317189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:46.317189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:46.317253 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:46.317431 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:46.317432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:46.317432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:46.317547 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:46.317701 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:46.317802 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 02:09:46.317873 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:09:46.319857 6969 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:09:46.321676 6969 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:09:46.328660 7494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.324282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57110 (local address 127.5.9.129:36689)
0809 02:09:46.324759 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:46.324764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:46.324775 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:09:46.324819 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:46.324822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:46.324849 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:46.324942 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:46.325152 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.326070 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.327580 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.328156 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.328257 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.328324 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:46.328504 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 02:09:46.328545 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":101,"threads_started":1}
I20260809 02:09:46.328665 7493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.324161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:36689 (local address 127.0.0.1:57110)
0809 02:09:46.324481 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:46.324492 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:46.324551 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:46.324973 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:46.324977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:46.324980 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:46.325061 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.325064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.326166 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.326167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.327484 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.327487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.328261 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.328262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.328367 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.328382 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 02:09:46.328484 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 02:09:46.328545 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":86,"threads_started":1}
I20260809 02:09:46.328895 7492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.324160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:39633 (local address 127.0.0.1:43310)
0809 02:09:46.324273 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:46.324315 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:46.324392 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:46.324990 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:46.324992 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:46.324994 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:46.325080 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.325082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.326454 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.326455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.327508 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.327509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.328428 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.328429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.328500 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.328505 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:09:46.328537 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 02:09:46.328583 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":64,"threads_started":1}
I20260809 02:09:46.329058 7495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.324374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:36783 (local address 127.0.0.1:52182)
0809 02:09:46.325041 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:46.325052 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:46.325103 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:46.325328 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:46.325329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:46.325332 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:46.325366 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.325368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.326711 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.326713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.327690 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.327693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.328656 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.328657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.328719 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.328722 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:09:46.328772 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 02:09:46.328819 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":82,"threads_started":1}
I20260809 02:09:46.329145 7496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.324502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52182 (local address 127.5.9.130:36783)
0809 02:09:46.324740 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:46.324746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:46.325165 (+ 419us) server_negotiation.cc:407] Connection header received
0809 02:09:46.325211 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:46.325214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:46.325228 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:46.325320 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:46.325430 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.326507 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.327783 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.328503 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.328567 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.328622 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:46.328933 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 02:09:46.328972 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":106,"threads_started":1}
I20260809 02:09:46.329473 7090 tablet_service.cc:1467] Processing CreateTablet for tablet 030a4377387d47ebb93678e69e440366 (DEFAULT_TABLE table=test-table [id=e88d9309f5e644479ff5778565b64019]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:46.329751 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.324530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43310 (local address 127.5.9.131:39633)
0809 02:09:46.324790 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:46.324793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:46.324829 (+ 36us) server_negotiation.cc:407] Connection header received
0809 02:09:46.324867 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:46.324870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:46.324886 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:46.325162 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:46.325223 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.326366 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.327601 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.328305 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.328707 (+ 402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.329488 (+ 781us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:46.329518 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 02:09:46.329560 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":165,"threads_started":1}
I20260809 02:09:46.329931 7090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 030a4377387d47ebb93678e69e440366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:46.329870 7259 tablet_service.cc:1467] Processing CreateTablet for tablet 030a4377387d47ebb93678e69e440366 (DEFAULT_TABLE table=test-table [id=e88d9309f5e644479ff5778565b64019]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:46.330163 7259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 030a4377387d47ebb93678e69e440366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:46.332856 7498 tablet_bootstrap.cc:492] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094: Bootstrap starting.
I20260809 02:09:46.333348 7499 tablet_bootstrap.cc:492] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee: Bootstrap starting.
I20260809 02:09:46.333576 7498 tablet_bootstrap.cc:654] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:46.333515 7404 tablet_service.cc:1467] Processing CreateTablet for tablet 030a4377387d47ebb93678e69e440366 (DEFAULT_TABLE table=test-table [id=e88d9309f5e644479ff5778565b64019]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:46.333892 7404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 030a4377387d47ebb93678e69e440366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:46.333990 7498 log.cc:824] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:46.334066 7499 tablet_bootstrap.cc:654] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:46.334401 7499 log.cc:824] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:46.335186 7499 tablet_bootstrap.cc:492] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee: No bootstrap required, opened a new log
I20260809 02:09:46.335309 7499 ts_tablet_manager.cc:1397] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 02:09:46.335634 7498 tablet_bootstrap.cc:492] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094: No bootstrap required, opened a new log
I20260809 02:09:46.335690 7498 ts_tablet_manager.cc:1397] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 02:09:46.337443 7503 tablet_bootstrap.cc:492] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d: Bootstrap starting.
I20260809 02:09:46.337706 7498 raft_consensus.cc:348] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } }
I20260809 02:09:46.337792 7499 raft_consensus.cc:348] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } }
I20260809 02:09:46.337908 7498 raft_consensus.cc:374] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:46.337946 7499 raft_consensus.cc:374] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:46.337958 7498 raft_consensus.cc:729] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ff7069fc3f4a0fa8a472de7ba60094, State: Initialized, Role: FOLLOWER
I20260809 02:09:46.337990 7499 raft_consensus.cc:729] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821f1f626975452a92167f161383dbee, State: Initialized, Role: FOLLOWER
I20260809 02:09:46.338116 7499 consensus_queue.cc:260] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } }
I20260809 02:09:46.338104 7498 consensus_queue.cc:260] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } }
I20260809 02:09:46.338411 7499 ts_tablet_manager.cc:1428] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:09:46.338421 7498 ts_tablet_manager.cc:1428] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 02:09:46.338515 7503 tablet_bootstrap.cc:654] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:46.338568 7182 heartbeater.cc:503] Master 127.5.9.190:35481 was elected leader, sending a full tablet report...
I20260809 02:09:46.338608 7328 heartbeater.cc:503] Master 127.5.9.190:35481 was elected leader, sending a full tablet report...
I20260809 02:09:46.338886 7503 log.cc:824] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:46.339773 7503 tablet_bootstrap.cc:492] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d: No bootstrap required, opened a new log
I20260809 02:09:46.339826 7503 ts_tablet_manager.cc:1397] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:09:46.341122 7503 raft_consensus.cc:348] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } }
I20260809 02:09:46.341251 7503 raft_consensus.cc:374] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:46.341274 7503 raft_consensus.cc:729] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a33c8858a824442f9fd3fc9805cc318d, State: Initialized, Role: FOLLOWER
I20260809 02:09:46.341390 7503 consensus_queue.cc:260] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } }
I20260809 02:09:46.341677 7503 ts_tablet_manager.cc:1428] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:09:46.341702 7475 heartbeater.cc:503] Master 127.5.9.190:35481 was elected leader, sending a full tablet report...
I20260809 02:09:46.344893 7507 raft_consensus.cc:482] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:09:46.344974 7507 raft_consensus.cc:504] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } }
I20260809 02:09:46.345307 7507 leader_election.cc:288] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30ff7069fc3f4a0fa8a472de7ba60094 (127.5.9.129:36689), 821f1f626975452a92167f161383dbee (127.5.9.130:36783)
I20260809 02:09:46.349210 7496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.345584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:34693 (local address 127.5.9.130:36783)
0809 02:09:46.345653 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:46.345655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:46.345664 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:09:46.345736 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:46.345737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:46.345747 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:46.345828 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:46.345979 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.346774 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.348274 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.348828 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.348901 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.348912 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:46.349102 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 02:09:46.349155 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:09:46.349239 7363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.345474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:36783 (local address 127.5.9.131:34693)
0809 02:09:46.345584 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:46.345596 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:46.345665 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:46.345838 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:46.345839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:46.345841 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:46.345911 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.345912 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.346886 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.346887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.348179 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.348183 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.348948 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.348949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.349032 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.349037 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:09:46.349095 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 02:09:46.349145 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":47}
I20260809 02:09:46.349246 7508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.345490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:36689 (local address 127.5.9.131:60621)
0809 02:09:46.345716 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:46.345726 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:46.345777 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:46.345915 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:46.345916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:46.345917 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:46.345982 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.345983 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.346912 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.346913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.348179 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.348183 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.348948 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.348949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.349032 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.349037 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:09:46.349093 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 02:09:46.349145 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":43,"thread_start_us":107,"threads_started":1}
I20260809 02:09:46.349485 7494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.345570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:60621 (local address 127.5.9.129:36689)
0809 02:09:46.345645 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:46.345647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:46.345787 (+ 140us) server_negotiation.cc:407] Connection header received
0809 02:09:46.345850 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:46.345851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:46.345864 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:46.345914 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:46.346043 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.346773 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.348274 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.348836 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.348901 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.348912 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:46.349332 (+ 420us) server_negotiation.cc:299] Negotiation successful
0809 02:09:46.349401 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:09:46.349723 7280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "030a4377387d47ebb93678e69e440366" candidate_uuid: "a33c8858a824442f9fd3fc9805cc318d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821f1f626975452a92167f161383dbee" is_pre_election: true
I20260809 02:09:46.349759 7132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "030a4377387d47ebb93678e69e440366" candidate_uuid: "a33c8858a824442f9fd3fc9805cc318d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" is_pre_election: true
I20260809 02:09:46.349900 7280 raft_consensus.cc:2454] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a33c8858a824442f9fd3fc9805cc318d in term 0.
I20260809 02:09:46.349920 7132 raft_consensus.cc:2454] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a33c8858a824442f9fd3fc9805cc318d in term 0.
I20260809 02:09:46.350126 7358 leader_election.cc:302] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821f1f626975452a92167f161383dbee, a33c8858a824442f9fd3fc9805cc318d; no voters:
I20260809 02:09:46.350260 7507 raft_consensus.cc:2781] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:09:46.350334 7507 raft_consensus.cc:482] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:09:46.350353 7507 raft_consensus.cc:3037] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:46.351101 7507 raft_consensus.cc:504] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } }
I20260809 02:09:46.351248 7507 leader_election.cc:288] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [CANDIDATE]: Term 1 election: Requested vote from peers 30ff7069fc3f4a0fa8a472de7ba60094 (127.5.9.129:36689), 821f1f626975452a92167f161383dbee (127.5.9.130:36783)
I20260809 02:09:46.351405 7280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "030a4377387d47ebb93678e69e440366" candidate_uuid: "a33c8858a824442f9fd3fc9805cc318d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821f1f626975452a92167f161383dbee"
I20260809 02:09:46.351418 7132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "030a4377387d47ebb93678e69e440366" candidate_uuid: "a33c8858a824442f9fd3fc9805cc318d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ff7069fc3f4a0fa8a472de7ba60094"
I20260809 02:09:46.351501 7280 raft_consensus.cc:3037] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:46.351521 7132 raft_consensus.cc:3037] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:46.352231 7280 raft_consensus.cc:2454] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a33c8858a824442f9fd3fc9805cc318d in term 1.
I20260809 02:09:46.352294 7132 raft_consensus.cc:2454] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a33c8858a824442f9fd3fc9805cc318d in term 1.
I20260809 02:09:46.352432 7358 leader_election.cc:302] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821f1f626975452a92167f161383dbee, a33c8858a824442f9fd3fc9805cc318d; no voters:
I20260809 02:09:46.352535 7507 raft_consensus.cc:2781] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:46.352699 7507 raft_consensus.cc:686] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [term 1 LEADER]: Becoming Leader. State: Replica: a33c8858a824442f9fd3fc9805cc318d, State: Running, Role: LEADER
I20260809 02:09:46.352806 7507 consensus_queue.cc:237] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } }
I20260809 02:09:46.353735 6969 catalog_manager.cc:5495] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d reported cstate change: term changed from 0 to 1, leader changed from <none> to a33c8858a824442f9fd3fc9805cc318d (127.5.9.131). New cstate: current_term: 1 leader_uuid: "a33c8858a824442f9fd3fc9805cc318d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } health_report { overall_health: UNKNOWN } } }
W20260809 02:09:46.354624 7330 tablet.cc:2369] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 02:09:46 dist-test-slave-p3t5 krb5kdc[6923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241385, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:46.394178 7516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.386520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:35481 (local address 127.0.0.1:46564)
0809 02:09:46.386709 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:46.386720 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:46.386808 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:46.386939 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:46.386940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:46.387261 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:46.387314 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.387316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.387951 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.387952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.388472 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.388473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.389051 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.389054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.389105 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.389783 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:46.389787 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:46.391421 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:46.392972 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:46.392980 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:46.392982 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:46.393204 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:46.393402 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:46.393402 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:46.393403 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:46.393463 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:46.393879 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:46.393886 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:46.393935 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 02:09:46.393994 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 02:09:46.394098 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":117,"threads_started":1}
I20260809 02:09:46.394275 7020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.386585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46564 (local address 127.5.9.190:35481)
0809 02:09:46.386663 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:46.386666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:46.386810 (+ 144us) server_negotiation.cc:407] Connection header received
0809 02:09:46.386855 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:46.386856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:46.386866 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:46.386920 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:46.387385 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.387880 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.388577 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.388818 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.388889 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.391516 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:46.391517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:46.391517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:46.391525 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:46.392891 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:46.393272 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:46.393274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:46.393274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:46.393332 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:46.393556 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:46.393557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:46.393557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:46.393811 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:46.393876 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:46.394028 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 02:09:46.394153 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":48}
I20260809 02:09:46.396503 6969 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:46564:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 02:09:46.396711 6969 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:09:46.398381 7259 tablet_service.cc:1467] Processing CreateTablet for tablet 7819e79fe6db411aacd1b7edd1261d56 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a104d52196f94ed39198e9eb5b334b83]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:09:46.398393 7090 tablet_service.cc:1467] Processing CreateTablet for tablet 7819e79fe6db411aacd1b7edd1261d56 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a104d52196f94ed39198e9eb5b334b83]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:09:46.398592 7259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7819e79fe6db411aacd1b7edd1261d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:46.398593 7090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7819e79fe6db411aacd1b7edd1261d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:46.398648 7404 tablet_service.cc:1467] Processing CreateTablet for tablet 7819e79fe6db411aacd1b7edd1261d56 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a104d52196f94ed39198e9eb5b334b83]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:09:46.398898 7404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7819e79fe6db411aacd1b7edd1261d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:46.400204 7503 tablet_bootstrap.cc:492] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d: Bootstrap starting.
I20260809 02:09:46.400235 7498 tablet_bootstrap.cc:492] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094: Bootstrap starting.
I20260809 02:09:46.400251 7499 tablet_bootstrap.cc:492] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee: Bootstrap starting.
I20260809 02:09:46.400744 7503 tablet_bootstrap.cc:654] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:46.400938 7498 tablet_bootstrap.cc:654] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:46.401021 7499 tablet_bootstrap.cc:654] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:46.401697 7503 tablet_bootstrap.cc:492] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d: No bootstrap required, opened a new log
I20260809 02:09:46.401762 7503 ts_tablet_manager.cc:1397] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:09:46.401813 7499 tablet_bootstrap.cc:492] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee: No bootstrap required, opened a new log
I20260809 02:09:46.401871 7499 ts_tablet_manager.cc:1397] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 02:09:46.401962 7498 tablet_bootstrap.cc:492] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094: No bootstrap required, opened a new log
I20260809 02:09:46.401947 7503 raft_consensus.cc:348] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.402012 7498 ts_tablet_manager.cc:1397] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 02:09:46.402041 7503 raft_consensus.cc:374] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:46.402076 7503 raft_consensus.cc:729] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a33c8858a824442f9fd3fc9805cc318d, State: Initialized, Role: FOLLOWER
I20260809 02:09:46.402050 7499 raft_consensus.cc:348] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.402144 7499 raft_consensus.cc:374] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:46.402132 7503 consensus_queue.cc:260] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.402172 7499 raft_consensus.cc:729] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821f1f626975452a92167f161383dbee, State: Initialized, Role: FOLLOWER
I20260809 02:09:46.402164 7498 raft_consensus.cc:348] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.402221 7503 ts_tablet_manager.cc:1428] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:09:46.402247 7498 raft_consensus.cc:374] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:46.402240 7499 consensus_queue.cc:260] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.402276 7498 raft_consensus.cc:729] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ff7069fc3f4a0fa8a472de7ba60094, State: Initialized, Role: FOLLOWER
I20260809 02:09:46.402324 7498 consensus_queue.cc:260] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.402335 7499 ts_tablet_manager.cc:1428] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 02:09:46.402412 7498 ts_tablet_manager.cc:1428] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:46.402508 7504 tablet_replica.cc:439] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } }
I20260809 02:09:46.402508 7505 tablet_replica.cc:439] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } }
I20260809 02:09:46.402606 7505 tablet_replica.cc:442] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:09:46.402607 7504 tablet_replica.cc:442] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:09:46.403267 7507 tablet_replica.cc:439] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } }
I20260809 02:09:46.403371 7507 tablet_replica.cc:442] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 02:09:46.432577 7183 tablet.cc:2369] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:09:46.537225 7476 tablet.cc:2369] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:09:46.542109 7507 raft_consensus.cc:482] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:09:46.542240 7507 raft_consensus.cc:504] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.542464 7507 leader_election.cc:288] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30ff7069fc3f4a0fa8a472de7ba60094 (127.5.9.129:36689), 821f1f626975452a92167f161383dbee (127.5.9.130:36783)
I20260809 02:09:46.542719 7132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7819e79fe6db411aacd1b7edd1261d56" candidate_uuid: "a33c8858a824442f9fd3fc9805cc318d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" is_pre_election: true
I20260809 02:09:46.542821 7132 raft_consensus.cc:2454] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a33c8858a824442f9fd3fc9805cc318d in term 0.
I20260809 02:09:46.542754 7280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7819e79fe6db411aacd1b7edd1261d56" candidate_uuid: "a33c8858a824442f9fd3fc9805cc318d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821f1f626975452a92167f161383dbee" is_pre_election: true
I20260809 02:09:46.542862 7280 raft_consensus.cc:2454] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a33c8858a824442f9fd3fc9805cc318d in term 0.
I20260809 02:09:46.543025 7358 leader_election.cc:302] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821f1f626975452a92167f161383dbee, a33c8858a824442f9fd3fc9805cc318d; no voters:
I20260809 02:09:46.543136 7507 raft_consensus.cc:2781] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:09:46.543179 7507 raft_consensus.cc:482] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:09:46.543200 7507 raft_consensus.cc:3037] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:46.543879 7507 raft_consensus.cc:504] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.544015 7507 leader_election.cc:288] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [CANDIDATE]: Term 1 election: Requested vote from peers 30ff7069fc3f4a0fa8a472de7ba60094 (127.5.9.129:36689), 821f1f626975452a92167f161383dbee (127.5.9.130:36783)
I20260809 02:09:46.544185 7132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7819e79fe6db411aacd1b7edd1261d56" candidate_uuid: "a33c8858a824442f9fd3fc9805cc318d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ff7069fc3f4a0fa8a472de7ba60094"
I20260809 02:09:46.544210 7280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7819e79fe6db411aacd1b7edd1261d56" candidate_uuid: "a33c8858a824442f9fd3fc9805cc318d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821f1f626975452a92167f161383dbee"
I20260809 02:09:46.544273 7280 raft_consensus.cc:3037] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:46.544282 7132 raft_consensus.cc:3037] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:46.544971 7132 raft_consensus.cc:2454] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a33c8858a824442f9fd3fc9805cc318d in term 1.
I20260809 02:09:46.544977 7280 raft_consensus.cc:2454] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a33c8858a824442f9fd3fc9805cc318d in term 1.
I20260809 02:09:46.545212 7358 leader_election.cc:302] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821f1f626975452a92167f161383dbee, a33c8858a824442f9fd3fc9805cc318d; no voters:
I20260809 02:09:46.545305 7507 raft_consensus.cc:2781] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:46.545373 7507 raft_consensus.cc:686] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [term 1 LEADER]: Becoming Leader. State: Replica: a33c8858a824442f9fd3fc9805cc318d, State: Running, Role: LEADER
I20260809 02:09:46.545437 7507 consensus_queue.cc:237] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.545853 7509 tablet_replica.cc:439] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d: TxnStatusTablet state changed. Reason: New leader a33c8858a824442f9fd3fc9805cc318d. Latest consensus state: current_term: 1 leader_uuid: "a33c8858a824442f9fd3fc9805cc318d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } }
I20260809 02:09:46.545953 7509 tablet_replica.cc:442] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:09:46.546218 6968 catalog_manager.cc:5495] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d reported cstate change: term changed from 0 to 1, leader changed from <none> to a33c8858a824442f9fd3fc9805cc318d (127.5.9.131). New cstate: current_term: 1 leader_uuid: "a33c8858a824442f9fd3fc9805cc318d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } health_report { overall_health: HEALTHY } } }
I20260809 02:09:46.546522 7521 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 02:09:46.546677 7521 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 02:09:46.587532 7516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.584362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:39633 (local address 127.0.0.1:43324)
0809 02:09:46.584450 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:46.584465 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:46.584526 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:46.584728 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:46.584729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:46.584730 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:09:46.584782 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.584784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.585884 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.585885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.586488 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.586491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.587010 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.587011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.587061 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.587081 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:09:46.587383 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:09:46.587390 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 02:09:46.587419 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 02:09:46.587460 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 02:09:46.587571 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.584455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43324 (local address 127.5.9.131:39633)
0809 02:09:46.584529 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:46.584531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:46.584543 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:09:46.584608 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:46.584610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:46.584649 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:46.584717 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:09:46.584844 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.585763 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.586579 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.586923 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.587003 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.587178 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:09:46.587313 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:09:46.587401 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:46.587424 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 02:09:46.587452 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:09:46.589810 7132 raft_consensus.cc:1261] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 FOLLOWER]: Refusing update from remote peer a33c8858a824442f9fd3fc9805cc318d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:09:46.589810 7280 raft_consensus.cc:1261] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Refusing update from remote peer a33c8858a824442f9fd3fc9805cc318d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:09:46.590240 7509 consensus_queue.cc:1035] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:09:46.590415 7507 consensus_queue.cc:1035] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d [LEADER]: Connected to new peer: Peer: permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:09:46.591701 7505 tablet_replica.cc:439] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094: TxnStatusTablet state changed. Reason: New leader a33c8858a824442f9fd3fc9805cc318d. Latest consensus state: current_term: 1 leader_uuid: "a33c8858a824442f9fd3fc9805cc318d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } }
I20260809 02:09:46.591814 7505 tablet_replica.cc:442] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:09:46.592384 7504 tablet_replica.cc:439] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee: TxnStatusTablet state changed. Reason: New leader a33c8858a824442f9fd3fc9805cc318d. Latest consensus state: current_term: 1 leader_uuid: "a33c8858a824442f9fd3fc9805cc318d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } }
I20260809 02:09:46.592504 7504 tablet_replica.cc:442] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:09:46.593041 7507 tablet_replica.cc:439] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a33c8858a824442f9fd3fc9805cc318d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } }
I20260809 02:09:46.593144 7507 tablet_replica.cc:442] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:09:46.593598 7509 tablet_replica.cc:439] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a33c8858a824442f9fd3fc9805cc318d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } }
I20260809 02:09:46.593729 7509 tablet_replica.cc:442] T 7819e79fe6db411aacd1b7edd1261d56 P a33c8858a824442f9fd3fc9805cc318d: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:09:46.593994 7504 tablet_replica.cc:439] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a33c8858a824442f9fd3fc9805cc318d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } }
I20260809 02:09:46.594080 7504 tablet_replica.cc:442] T 7819e79fe6db411aacd1b7edd1261d56 P 821f1f626975452a92167f161383dbee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:09:46.594852 7505 tablet_replica.cc:439] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a33c8858a824442f9fd3fc9805cc318d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } } }
I20260809 02:09:46.594961 7505 tablet_replica.cc:442] T 7819e79fe6db411aacd1b7edd1261d56 P 30ff7069fc3f4a0fa8a472de7ba60094: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:09:46.594955 7524 mvcc.cc:205] Tried to move back new op lower bound from 7316444719468367872 to 7316444719290793984. Current Snapshot: MvccSnapshot[applied={T|T < 7316444719468367872 or (T in {7316444719468367872})}]
I20260809 02:09:46.595425 7525 mvcc.cc:205] Tried to move back new op lower bound from 7316444719468367872 to 7316444719290793984. Current Snapshot: MvccSnapshot[applied={T|T < 7316444719468367872 or (T in {7316444719468367872})}]
I20260809 02:09:46.598182 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.595765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43330 (local address 127.5.9.131:39633)
0809 02:09:46.595805 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:46.595807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:46.595819 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:09:46.595926 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:46.595927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:46.595935 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:46.596007 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:09:46.596160 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.596645 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.597431 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.597692 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.597753 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.597929 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:09:46.597963 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:09:46.598019 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:46.598077 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 02:09:46.598099 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:09:46.603565 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.600930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:34557 (local address 127.5.9.131:39633)
0809 02:09:46.600967 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:46.600969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:46.601000 (+ 31us) server_negotiation.cc:407] Connection header received
0809 02:09:46.601070 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:46.601070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:46.601078 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:46.601123 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:09:46.601245 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.601756 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.602862 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.603042 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.603108 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.603323 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:09:46.603350 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:09:46.603390 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:46.603471 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 02:09:46.603488 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 02:09:46.603653 7481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:46.600876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:39633 (local address 127.5.9.131:34557)
0809 02:09:46.600956 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:46.600967 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:46.601014 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:46.601127 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:46.601127 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:46.601129 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:09:46.601191 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.601193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.601827 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.601828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.602759 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:46.602762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:46.603124 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:46.603125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:46.603195 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:46.603213 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:09:46.603412 (+ 199us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:09:46.603414 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:09:46.603470 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 02:09:46.603517 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 02:09:46.606207 7132 raft_consensus.cc:1261] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 FOLLOWER]: Refusing update from remote peer a33c8858a824442f9fd3fc9805cc318d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:09:46.606266 7280 raft_consensus.cc:1261] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Refusing update from remote peer a33c8858a824442f9fd3fc9805cc318d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:09:46.606457 7507 consensus_queue.cc:1035] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:09:46.606704 7507 consensus_queue.cc:1035] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [LEADER]: Connected to new peer: Peer: permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:09:46.608753 7526 mvcc.cc:205] Tried to move back new op lower bound from 7316444719537233920 to 7316444718501842944. Current Snapshot: MvccSnapshot[applied={T|T < 7316444719537233920}]
I20260809 02:09:46.622879 6966 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:09:46.623008 6966 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:09:46.623945 6966 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47: Sending DeleteTablet for 3 replicas of tablet 030a4377387d47ebb93678e69e440366
I20260809 02:09:46.624291 7403 tablet_service.cc:1514] Processing DeleteTablet for tablet 030a4377387d47ebb93678e69e440366 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:46 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:43310
I20260809 02:09:46.624303 7090 tablet_service.cc:1514] Processing DeleteTablet for tablet 030a4377387d47ebb93678e69e440366 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:46 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:57110
I20260809 02:09:46.624361 7259 tablet_service.cc:1514] Processing DeleteTablet for tablet 030a4377387d47ebb93678e69e440366 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:46 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:52182
I20260809 02:09:46.624698 7542 tablet_replica.cc:330] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094: stopping tablet replica
I20260809 02:09:46.624699 7540 tablet_replica.cc:330] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee: stopping tablet replica
I20260809 02:09:46.624842 7542 raft_consensus.cc:2229] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:09:46.624873 7540 raft_consensus.cc:2229] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:09:46.624940 7542 raft_consensus.cc:2258] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:46.624998 7540 raft_consensus.cc:2258] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:46.625058 6966 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:09:46.625221 6966 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:09:46.625171 7541 tablet_replica.cc:330] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d: stopping tablet replica
I20260809 02:09:46.625438 7541 raft_consensus.cc:2229] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:09:46.626371 7542 ts_tablet_manager.cc:1905] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:46.626822 7090 tablet_service.cc:1467] Processing CreateTablet for tablet 6651ce7820744fd59f0603d89b91a1a3 (DEFAULT_TABLE table=test-table [id=f8af7461f35e4d3dbbe3ccdde95b82ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:46.626964 7090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6651ce7820744fd59f0603d89b91a1a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:46.628265 7542 ts_tablet_manager.cc:1918] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:09:46.628330 7542 log.cc:1196] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/wal/wals/030a4377387d47ebb93678e69e440366
I20260809 02:09:46.628432 7498 tablet_bootstrap.cc:492] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094: Bootstrap starting.
I20260809 02:09:46.628690 7259 tablet_service.cc:1467] Processing CreateTablet for tablet 6651ce7820744fd59f0603d89b91a1a3 (DEFAULT_TABLE table=test-table [id=f8af7461f35e4d3dbbe3ccdde95b82ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:46.628826 7259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6651ce7820744fd59f0603d89b91a1a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:46.628789 7542 ts_tablet_manager.cc:1939] T 030a4377387d47ebb93678e69e440366 P 30ff7069fc3f4a0fa8a472de7ba60094: Deleting consensus metadata
I20260809 02:09:46.628865 7498 tablet_bootstrap.cc:654] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:46.629333 6954 catalog_manager.cc:4841] TS 30ff7069fc3f4a0fa8a472de7ba60094 (127.5.9.129:36689): tablet 030a4377387d47ebb93678e69e440366 (table test-table [id=e88d9309f5e644479ff5778565b64019]) successfully deleted
I20260809 02:09:46.629549 7498 tablet_bootstrap.cc:492] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094: No bootstrap required, opened a new log
I20260809 02:09:46.629582 7498 ts_tablet_manager.cc:1397] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:09:46.629776 7498 raft_consensus.cc:348] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.629869 7498 raft_consensus.cc:374] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:46.629904 7498 raft_consensus.cc:729] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ff7069fc3f4a0fa8a472de7ba60094, State: Initialized, Role: FOLLOWER
I20260809 02:09:46.629863 7540 ts_tablet_manager.cc:1905] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:46.629978 7498 consensus_queue.cc:260] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.630076 7498 ts_tablet_manager.cc:1428] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:09:46.630301 7499 tablet_bootstrap.cc:492] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee: Bootstrap starting.
I20260809 02:09:46.630746 7499 tablet_bootstrap.cc:654] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:46.631763 7540 ts_tablet_manager.cc:1918] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:09:46.631817 7499 tablet_bootstrap.cc:492] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee: No bootstrap required, opened a new log
I20260809 02:09:46.631865 7540 log.cc:1196] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/030a4377387d47ebb93678e69e440366
I20260809 02:09:46.631888 7499 ts_tablet_manager.cc:1397] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 02:09:46.632042 7499 raft_consensus.cc:348] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.632125 7499 raft_consensus.cc:374] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:46.632144 7540 ts_tablet_manager.cc:1939] T 030a4377387d47ebb93678e69e440366 P 821f1f626975452a92167f161383dbee: Deleting consensus metadata
I20260809 02:09:46.632148 7499 raft_consensus.cc:729] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821f1f626975452a92167f161383dbee, State: Initialized, Role: FOLLOWER
I20260809 02:09:46.632243 7499 consensus_queue.cc:260] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.632349 7499 ts_tablet_manager.cc:1428] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 02:09:46.626811 7403 tablet_service.cc:1467] Processing CreateTablet for tablet 6651ce7820744fd59f0603d89b91a1a3 (DEFAULT_TABLE table=test-table [id=f8af7461f35e4d3dbbe3ccdde95b82ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:46.632633 6956 catalog_manager.cc:4841] TS 821f1f626975452a92167f161383dbee (127.5.9.130:36783): tablet 030a4377387d47ebb93678e69e440366 (table test-table [id=e88d9309f5e644479ff5778565b64019]) successfully deleted
I20260809 02:09:46.632669 7403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6651ce7820744fd59f0603d89b91a1a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:46.633777 7541 raft_consensus.cc:2258] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:46.634390 7541 ts_tablet_manager.cc:1905] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:46.634440 7503 tablet_bootstrap.cc:492] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d: Bootstrap starting.
I20260809 02:09:46.634895 7503 tablet_bootstrap.cc:654] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:46.635653 7503 tablet_bootstrap.cc:492] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d: No bootstrap required, opened a new log
I20260809 02:09:46.635711 7503 ts_tablet_manager.cc:1397] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:46.635874 7503 raft_consensus.cc:348] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.635958 7503 raft_consensus.cc:374] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:46.635978 7541 ts_tablet_manager.cc:1918] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:09:46.636101 7541 log.cc:1196] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/030a4377387d47ebb93678e69e440366
I20260809 02:09:46.635982 7503 raft_consensus.cc:729] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a33c8858a824442f9fd3fc9805cc318d, State: Initialized, Role: FOLLOWER
I20260809 02:09:46.636298 7503 consensus_queue.cc:260] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.636409 7541 ts_tablet_manager.cc:1939] T 030a4377387d47ebb93678e69e440366 P a33c8858a824442f9fd3fc9805cc318d: Deleting consensus metadata
I20260809 02:09:46.636456 7503 ts_tablet_manager.cc:1428] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:46.636878 6956 catalog_manager.cc:4841] TS a33c8858a824442f9fd3fc9805cc318d (127.5.9.131:39633): tablet 030a4377387d47ebb93678e69e440366 (table test-table [id=e88d9309f5e644479ff5778565b64019]) successfully deleted
I20260809 02:09:46.727355 7507 raft_consensus.cc:482] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:09:46.727506 7507 raft_consensus.cc:504] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.727675 7507 leader_election.cc:288] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30ff7069fc3f4a0fa8a472de7ba60094 (127.5.9.129:36689), 821f1f626975452a92167f161383dbee (127.5.9.130:36783)
I20260809 02:09:46.728017 7132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6651ce7820744fd59f0603d89b91a1a3" candidate_uuid: "a33c8858a824442f9fd3fc9805cc318d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" is_pre_election: true
I20260809 02:09:46.728034 7280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6651ce7820744fd59f0603d89b91a1a3" candidate_uuid: "a33c8858a824442f9fd3fc9805cc318d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821f1f626975452a92167f161383dbee" is_pre_election: true
I20260809 02:09:46.728142 7132 raft_consensus.cc:2454] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a33c8858a824442f9fd3fc9805cc318d in term 0.
I20260809 02:09:46.728152 7280 raft_consensus.cc:2454] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a33c8858a824442f9fd3fc9805cc318d in term 0.
I20260809 02:09:46.728325 7358 leader_election.cc:302] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821f1f626975452a92167f161383dbee, a33c8858a824442f9fd3fc9805cc318d; no voters:
I20260809 02:09:46.728459 7507 raft_consensus.cc:2781] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:09:46.728520 7507 raft_consensus.cc:482] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:09:46.728536 7507 raft_consensus.cc:3037] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:46.729251 7507 raft_consensus.cc:504] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.729408 7507 leader_election.cc:288] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [CANDIDATE]: Term 1 election: Requested vote from peers 30ff7069fc3f4a0fa8a472de7ba60094 (127.5.9.129:36689), 821f1f626975452a92167f161383dbee (127.5.9.130:36783)
I20260809 02:09:46.729562 7132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6651ce7820744fd59f0603d89b91a1a3" candidate_uuid: "a33c8858a824442f9fd3fc9805cc318d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ff7069fc3f4a0fa8a472de7ba60094"
I20260809 02:09:46.729595 7280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6651ce7820744fd59f0603d89b91a1a3" candidate_uuid: "a33c8858a824442f9fd3fc9805cc318d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821f1f626975452a92167f161383dbee"
I20260809 02:09:46.729704 7280 raft_consensus.cc:3037] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:46.729708 7132 raft_consensus.cc:3037] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:46.730448 7132 raft_consensus.cc:2454] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a33c8858a824442f9fd3fc9805cc318d in term 1.
I20260809 02:09:46.730566 7280 raft_consensus.cc:2454] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a33c8858a824442f9fd3fc9805cc318d in term 1.
I20260809 02:09:46.730677 7356 leader_election.cc:302] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ff7069fc3f4a0fa8a472de7ba60094, a33c8858a824442f9fd3fc9805cc318d; no voters:
I20260809 02:09:46.730762 7507 raft_consensus.cc:2781] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:46.730829 7507 raft_consensus.cc:686] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [term 1 LEADER]: Becoming Leader. State: Replica: a33c8858a824442f9fd3fc9805cc318d, State: Running, Role: LEADER
I20260809 02:09:46.730883 7507 consensus_queue.cc:237] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:46.731604 6966 catalog_manager.cc:5495] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d reported cstate change: term changed from 0 to 1, leader changed from <none> to a33c8858a824442f9fd3fc9805cc318d (127.5.9.131). New cstate: current_term: 1 leader_uuid: "a33c8858a824442f9fd3fc9805cc318d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } health_report { overall_health: HEALTHY } } }
I20260809 02:09:46.753216 7132 raft_consensus.cc:1261] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 FOLLOWER]: Refusing update from remote peer a33c8858a824442f9fd3fc9805cc318d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:09:46.753290 7280 raft_consensus.cc:1261] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Refusing update from remote peer a33c8858a824442f9fd3fc9805cc318d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:09:46.753717 7537 consensus_queue.cc:1035] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [LEADER]: Connected to new peer: Peer: permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:09:46.753877 7507 consensus_queue.cc:1035] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:09:46.759197 7524 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 02:09:48.278887 6967 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:09:48.279045 6967 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:09:48.279893 6967 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47: Sending DeleteTablet for 3 replicas of tablet 6651ce7820744fd59f0603d89b91a1a3
I20260809 02:09:48.280174 7090 tablet_service.cc:1514] Processing DeleteTablet for tablet 6651ce7820744fd59f0603d89b91a1a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:48 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:57110
I20260809 02:09:48.280251 7404 tablet_service.cc:1514] Processing DeleteTablet for tablet 6651ce7820744fd59f0603d89b91a1a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:48 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:43310
I20260809 02:09:48.280314 7259 tablet_service.cc:1514] Processing DeleteTablet for tablet 6651ce7820744fd59f0603d89b91a1a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:48 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:52182
I20260809 02:09:48.280504 7570 tablet_replica.cc:330] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee: stopping tablet replica
I20260809 02:09:48.280570 7570 raft_consensus.cc:2229] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:09:48.280601 7570 raft_consensus.cc:2258] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:48.280892 7570 ts_tablet_manager.cc:1905] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:48.282480 7570 ts_tablet_manager.cc:1918] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:09:48.282562 7570 log.cc:1196] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/6651ce7820744fd59f0603d89b91a1a3
I20260809 02:09:48.282815 7570 ts_tablet_manager.cc:1939] T 6651ce7820744fd59f0603d89b91a1a3 P 821f1f626975452a92167f161383dbee: Deleting consensus metadata
I20260809 02:09:48.283224 6956 catalog_manager.cc:4841] TS 821f1f626975452a92167f161383dbee (127.5.9.130:36783): tablet 6651ce7820744fd59f0603d89b91a1a3 (table test-table [id=f8af7461f35e4d3dbbe3ccdde95b82ab]) successfully deleted
I20260809 02:09:48.283509 7568 tablet_replica.cc:330] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094: stopping tablet replica
I20260809 02:09:48.283600 7568 raft_consensus.cc:2229] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:09:48.283655 7568 raft_consensus.cc:2258] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:48.283785 7569 tablet_replica.cc:330] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d: stopping tablet replica
I20260809 02:09:48.283869 7569 raft_consensus.cc:2229] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:09:48.283950 7569 raft_consensus.cc:2258] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:48.283927 7568 ts_tablet_manager.cc:1905] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:48.284214 7569 ts_tablet_manager.cc:1905] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:48.285887 7569 ts_tablet_manager.cc:1918] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:09:48.286011 7569 log.cc:1196] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/6651ce7820744fd59f0603d89b91a1a3
I20260809 02:09:48.286012 7568 ts_tablet_manager.cc:1918] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:09:48.286155 7568 log.cc:1196] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/wal/wals/6651ce7820744fd59f0603d89b91a1a3
I20260809 02:09:48.286310 7569 ts_tablet_manager.cc:1939] T 6651ce7820744fd59f0603d89b91a1a3 P a33c8858a824442f9fd3fc9805cc318d: Deleting consensus metadata
I20260809 02:09:48.286451 7568 ts_tablet_manager.cc:1939] T 6651ce7820744fd59f0603d89b91a1a3 P 30ff7069fc3f4a0fa8a472de7ba60094: Deleting consensus metadata
I20260809 02:09:48.286756 6956 catalog_manager.cc:4841] TS a33c8858a824442f9fd3fc9805cc318d (127.5.9.131:39633): tablet 6651ce7820744fd59f0603d89b91a1a3 (table test-table [id=f8af7461f35e4d3dbbe3ccdde95b82ab]) successfully deleted
I20260809 02:09:48.286906 6954 catalog_manager.cc:4841] TS 30ff7069fc3f4a0fa8a472de7ba60094 (127.5.9.129:36689): tablet 6651ce7820744fd59f0603d89b91a1a3 (table test-table [id=f8af7461f35e4d3dbbe3ccdde95b82ab]) successfully deleted
Aug 09 02:09:48 dist-test-slave-p3t5 krb5kdc[6923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241388, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:09:48 dist-test-slave-p3t5 krb5kdc[6923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241388, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:48.315029 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:48.301154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46572 (local address 127.5.9.190:35481)
0809 02:09:48.301334 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:48.301336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:48.301346 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:09:48.301424 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:48.301425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:48.301437 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:48.301501 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:48.302323 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.302890 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.303786 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.304031 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.304136 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:48.312071 (+ 7935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:48.312073 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:48.312074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:48.312083 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:48.313582 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:48.314081 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:48.314083 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:48.314083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:48.314164 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:48.314404 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:48.314405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:48.314406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:48.314544 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:48.314664 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:48.314802 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 02:09:48.314907 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":120,"threads_started":1}
I20260809 02:09:48.316874 6967 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:09:48.317096 6967 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:09:48.318558 7090 tablet_service.cc:1467] Processing CreateTablet for tablet 5a55eec1ce7346dbbaf9046bf3e02410 (DEFAULT_TABLE table=test-table [id=d7ca457ef06f4b9ea995ed3818697fbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:48.318614 7404 tablet_service.cc:1467] Processing CreateTablet for tablet 5a55eec1ce7346dbbaf9046bf3e02410 (DEFAULT_TABLE table=test-table [id=d7ca457ef06f4b9ea995ed3818697fbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:48.318598 7259 tablet_service.cc:1467] Processing CreateTablet for tablet 5a55eec1ce7346dbbaf9046bf3e02410 (DEFAULT_TABLE table=test-table [id=d7ca457ef06f4b9ea995ed3818697fbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:48.318734 7404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a55eec1ce7346dbbaf9046bf3e02410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:48.318737 7090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a55eec1ce7346dbbaf9046bf3e02410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:48.318778 7259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a55eec1ce7346dbbaf9046bf3e02410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:48.319880 7580 tablet_bootstrap.cc:492] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee: Bootstrap starting.
I20260809 02:09:48.320323 7580 tablet_bootstrap.cc:654] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:48.320433 7582 tablet_bootstrap.cc:492] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d: Bootstrap starting.
I20260809 02:09:48.320796 7582 tablet_bootstrap.cc:654] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:48.321018 7580 tablet_bootstrap.cc:492] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee: No bootstrap required, opened a new log
I20260809 02:09:48.321085 7580 ts_tablet_manager.cc:1397] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:48.321187 7581 tablet_bootstrap.cc:492] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094: Bootstrap starting.
I20260809 02:09:48.321267 7580 raft_consensus.cc:348] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.321382 7580 raft_consensus.cc:374] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:48.321416 7580 raft_consensus.cc:729] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821f1f626975452a92167f161383dbee, State: Initialized, Role: FOLLOWER
I20260809 02:09:48.321475 7580 consensus_queue.cc:260] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.321620 7581 tablet_bootstrap.cc:654] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:48.321713 7580 ts_tablet_manager.cc:1428] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:09:48.322320 7582 tablet_bootstrap.cc:492] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d: No bootstrap required, opened a new log
I20260809 02:09:48.322381 7582 ts_tablet_manager.cc:1397] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:09:48.322510 7582 raft_consensus.cc:348] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.322620 7582 raft_consensus.cc:374] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:48.322643 7582 raft_consensus.cc:729] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a33c8858a824442f9fd3fc9805cc318d, State: Initialized, Role: FOLLOWER
I20260809 02:09:48.322697 7582 consensus_queue.cc:260] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.322799 7582 ts_tablet_manager.cc:1428] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:48.323028 7581 tablet_bootstrap.cc:492] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094: No bootstrap required, opened a new log
I20260809 02:09:48.323073 7581 ts_tablet_manager.cc:1397] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:09:48.323297 7581 raft_consensus.cc:348] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.323374 7581 raft_consensus.cc:374] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:48.323390 7581 raft_consensus.cc:729] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ff7069fc3f4a0fa8a472de7ba60094, State: Initialized, Role: FOLLOWER
I20260809 02:09:48.323501 7581 consensus_queue.cc:260] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.323725 7581 ts_tablet_manager.cc:1428] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 02:09:48.410915 7587 raft_consensus.cc:482] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:09:48.411036 7587 raft_consensus.cc:504] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.411435 7587 leader_election.cc:288] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821f1f626975452a92167f161383dbee (127.5.9.130:36783), a33c8858a824442f9fd3fc9805cc318d (127.5.9.131:39633)
I20260809 02:09:48.415165 7589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:48.411764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:55831 (local address 127.5.9.130:36783)
0809 02:09:48.412081 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:48.412083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:48.412094 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:09:48.412182 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:48.412183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:48.412200 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:48.412285 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:48.412464 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.413211 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.414328 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.414740 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.414833 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:48.414845 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:48.415036 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 02:09:48.415077 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":233,"threads_started":1}
I20260809 02:09:48.415165 7588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:48.411695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:36783 (local address 127.5.9.129:55831)
0809 02:09:48.412015 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:48.412029 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:48.412109 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:48.412291 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:48.412291 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:48.412294 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:48.412385 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:48.412387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.413294 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.413295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:48.414225 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:48.414228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.414862 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.414864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:48.414944 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:48.414948 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:09:48.415016 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 02:09:48.415062 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":109,"threads_started":1}
I20260809 02:09:48.415264 7591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:48.412008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:40473 (local address 127.5.9.131:39633)
0809 02:09:48.412157 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:48.412159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:48.412171 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:09:48.412203 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:48.412204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:48.412214 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:48.412306 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:09:48.412887 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.413472 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.414480 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.414875 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.414963 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:48.414975 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:48.415133 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 02:09:48.415179 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
I20260809 02:09:48.415401 7590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:48.411933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:39633 (local address 127.5.9.129:40473)
0809 02:09:48.412079 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:48.412090 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:48.412148 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:48.412312 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:48.412313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:48.412315 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:09:48.412775 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:48.412778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.413587 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.413588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:48.414378 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:48.414380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.414965 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.414966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:48.415048 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:48.415058 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 02:09:48.415127 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 02:09:48.415171 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260809 02:09:48.415692 7428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a55eec1ce7346dbbaf9046bf3e02410" candidate_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a33c8858a824442f9fd3fc9805cc318d" is_pre_election: true
I20260809 02:09:48.415885 7428 raft_consensus.cc:2454] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ff7069fc3f4a0fa8a472de7ba60094 in term 0.
I20260809 02:09:48.415941 7280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a55eec1ce7346dbbaf9046bf3e02410" candidate_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821f1f626975452a92167f161383dbee" is_pre_election: true
I20260809 02:09:48.416057 7280 raft_consensus.cc:2454] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ff7069fc3f4a0fa8a472de7ba60094 in term 0.
I20260809 02:09:48.416132 7061 leader_election.cc:302] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ff7069fc3f4a0fa8a472de7ba60094, a33c8858a824442f9fd3fc9805cc318d; no voters:
I20260809 02:09:48.416325 7587 raft_consensus.cc:2781] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:09:48.416379 7587 raft_consensus.cc:482] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:09:48.416395 7587 raft_consensus.cc:3037] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:48.417156 7587 raft_consensus.cc:504] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.417310 7587 leader_election.cc:288] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [CANDIDATE]: Term 1 election: Requested vote from peers 821f1f626975452a92167f161383dbee (127.5.9.130:36783), a33c8858a824442f9fd3fc9805cc318d (127.5.9.131:39633)
I20260809 02:09:48.417516 7428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a55eec1ce7346dbbaf9046bf3e02410" candidate_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a33c8858a824442f9fd3fc9805cc318d"
I20260809 02:09:48.417500 7280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a55eec1ce7346dbbaf9046bf3e02410" candidate_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821f1f626975452a92167f161383dbee"
I20260809 02:09:48.417590 7280 raft_consensus.cc:3037] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:48.417590 7428 raft_consensus.cc:3037] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:48.418437 7280 raft_consensus.cc:2454] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30ff7069fc3f4a0fa8a472de7ba60094 in term 1.
I20260809 02:09:48.418437 7428 raft_consensus.cc:2454] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30ff7069fc3f4a0fa8a472de7ba60094 in term 1.
I20260809 02:09:48.418632 7061 leader_election.cc:302] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ff7069fc3f4a0fa8a472de7ba60094, a33c8858a824442f9fd3fc9805cc318d; no voters:
I20260809 02:09:48.418730 7587 raft_consensus.cc:2781] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:48.418895 7587 raft_consensus.cc:686] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 LEADER]: Becoming Leader. State: Replica: 30ff7069fc3f4a0fa8a472de7ba60094, State: Running, Role: LEADER
I20260809 02:09:48.418980 7587 consensus_queue.cc:237] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.419763 6966 catalog_manager.cc:5495] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30ff7069fc3f4a0fa8a472de7ba60094 (127.5.9.129). New cstate: current_term: 1 leader_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } health_report { overall_health: UNKNOWN } } }
I20260809 02:09:48.443732 7595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:48.441054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57116 (local address 127.5.9.129:36689)
0809 02:09:48.441248 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:48.441250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:48.441257 (+ 7us) server_negotiation.cc:407] Connection header received
0809 02:09:48.441298 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:48.441298 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:48.441318 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:48.441401 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:09:48.441521 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.442125 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.442898 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.443107 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.443163 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:48.443353 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:09:48.443476 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:09:48.443533 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:48.443573 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 02:09:48.443590 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":101,"threads_started":1}
I20260809 02:09:48.449097 7591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:48.446652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:50581 (local address 127.5.9.131:39633)
0809 02:09:48.446753 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:48.446755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:48.446815 (+ 60us) server_negotiation.cc:407] Connection header received
0809 02:09:48.446866 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:48.446867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:48.446878 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:48.446933 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:09:48.447104 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.447604 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.448438 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.448634 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.448687 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:48.448883 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:09:48.448916 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:09:48.448956 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:48.449009 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 02:09:48.449029 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 02:09:48.449129 7598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:48.446550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:39633 (local address 127.5.9.129:50581)
0809 02:09:48.446722 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:48.446748 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:48.446817 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:48.446946 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:48.446947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:48.446948 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:09:48.447006 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:48.447007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.447677 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.447677 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:48.448355 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:48.448358 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.448708 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.448709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:48.448767 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:48.448783 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:09:48.448971 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:09:48.448972 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 02:09:48.449008 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 02:09:48.449044 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
I20260809 02:09:48.451650 7280 raft_consensus.cc:1261] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Refusing update from remote peer 30ff7069fc3f4a0fa8a472de7ba60094: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:09:48.451683 7428 raft_consensus.cc:1261] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d [term 1 FOLLOWER]: Refusing update from remote peer 30ff7069fc3f4a0fa8a472de7ba60094: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:09:48.452008 7587 consensus_queue.cc:1035] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:09:48.452165 7592 consensus_queue.cc:1035] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:09:48.460005 7595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:48.457491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:54967 (local address 127.5.9.129:36689)
0809 02:09:48.457542 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:48.457545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:48.457687 (+ 142us) server_negotiation.cc:407] Connection header received
0809 02:09:48.457783 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:48.457785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:48.457797 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:48.457855 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:09:48.457989 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.458495 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.459312 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.459568 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.459648 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:48.459772 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:09:48.459811 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:09:48.459865 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:48.459917 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 02:09:48.459941 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:09:48.460026 7602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:48.457426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:36689 (local address 127.5.9.131:54967)
0809 02:09:48.457606 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:48.457641 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:48.457700 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:48.457864 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:48.457865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:48.457867 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:09:48.457923 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:48.457924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.458590 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.458592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:48.459212 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:48.459215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:48.459626 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:48.459627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:48.459696 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:48.459703 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:09:48.459881 (+ 178us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:09:48.459882 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 02:09:48.459915 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 02:09:48.459958 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":91,"threads_started":1}
I20260809 02:09:48.470660 6967 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:09:48.470788 6967 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:09:48.471592 6967 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47: Sending DeleteTablet for 3 replicas of tablet 5a55eec1ce7346dbbaf9046bf3e02410
I20260809 02:09:48.471921 7259 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a55eec1ce7346dbbaf9046bf3e02410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:48 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:52182
I20260809 02:09:48.471935 7404 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a55eec1ce7346dbbaf9046bf3e02410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:48 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:43310
I20260809 02:09:48.472046 7570 tablet_replica.cc:330] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee: stopping tablet replica
I20260809 02:09:48.472067 7569 tablet_replica.cc:330] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d: stopping tablet replica
I20260809 02:09:48.472106 7570 raft_consensus.cc:2229] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:09:48.472115 7569 raft_consensus.cc:2229] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:09:48.472150 7570 raft_consensus.cc:2258] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:48.472152 7569 raft_consensus.cc:2258] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:48.472541 7570 ts_tablet_manager.cc:1905] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:48.472573 6967 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:09:48.472735 6967 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:09:48.473227 7569 ts_tablet_manager.cc:1905] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:48.474251 7570 ts_tablet_manager.cc:1918] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:09:48.474325 7570 log.cc:1196] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/5a55eec1ce7346dbbaf9046bf3e02410
I20260809 02:09:48.474587 7570 ts_tablet_manager.cc:1939] T 5a55eec1ce7346dbbaf9046bf3e02410 P 821f1f626975452a92167f161383dbee: Deleting consensus metadata
I20260809 02:09:48.474718 7569 ts_tablet_manager.cc:1918] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:09:48.474781 7569 log.cc:1196] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/5a55eec1ce7346dbbaf9046bf3e02410
I20260809 02:09:48.474949 6956 catalog_manager.cc:4841] TS 821f1f626975452a92167f161383dbee (127.5.9.130:36783): tablet 5a55eec1ce7346dbbaf9046bf3e02410 (table test-table [id=d7ca457ef06f4b9ea995ed3818697fbe]) successfully deleted
I20260809 02:09:48.474999 7569 ts_tablet_manager.cc:1939] T 5a55eec1ce7346dbbaf9046bf3e02410 P a33c8858a824442f9fd3fc9805cc318d: Deleting consensus metadata
I20260809 02:09:48.475054 7090 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a55eec1ce7346dbbaf9046bf3e02410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:48 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:57110
I20260809 02:09:48.475175 7568 tablet_replica.cc:330] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094: stopping tablet replica
I20260809 02:09:48.475227 7568 raft_consensus.cc:2229] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:09:48.475327 7568 raft_consensus.cc:2258] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:48.475392 6956 catalog_manager.cc:4841] TS a33c8858a824442f9fd3fc9805cc318d (127.5.9.131:39633): tablet 5a55eec1ce7346dbbaf9046bf3e02410 (table test-table [id=d7ca457ef06f4b9ea995ed3818697fbe]) successfully deleted
I20260809 02:09:48.475725 7568 ts_tablet_manager.cc:1905] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:48.476291 7259 tablet_service.cc:1467] Processing CreateTablet for tablet 820f9d72c45d45208ff09c408e924fa6 (DEFAULT_TABLE table=test-table [id=1ad08c29f64442a99c92b137186b31dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:48.476436 7259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 820f9d72c45d45208ff09c408e924fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:48.477283 7568 ts_tablet_manager.cc:1918] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:09:48.477322 7580 tablet_bootstrap.cc:492] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee: Bootstrap starting.
I20260809 02:09:48.477345 7568 log.cc:1196] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-0/wal/wals/5a55eec1ce7346dbbaf9046bf3e02410
I20260809 02:09:48.477381 7404 tablet_service.cc:1467] Processing CreateTablet for tablet 820f9d72c45d45208ff09c408e924fa6 (DEFAULT_TABLE table=test-table [id=1ad08c29f64442a99c92b137186b31dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:48.477488 7404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 820f9d72c45d45208ff09c408e924fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:48.477573 7568 ts_tablet_manager.cc:1939] T 5a55eec1ce7346dbbaf9046bf3e02410 P 30ff7069fc3f4a0fa8a472de7ba60094: Deleting consensus metadata
I20260809 02:09:48.477978 7090 tablet_service.cc:1467] Processing CreateTablet for tablet 820f9d72c45d45208ff09c408e924fa6 (DEFAULT_TABLE table=test-table [id=1ad08c29f64442a99c92b137186b31dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:09:48.478026 7580 tablet_bootstrap.cc:654] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:48.478096 7090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 820f9d72c45d45208ff09c408e924fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:48.479055 7582 tablet_bootstrap.cc:492] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d: Bootstrap starting.
I20260809 02:09:48.479298 7581 tablet_bootstrap.cc:492] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094: Bootstrap starting.
I20260809 02:09:48.479417 7582 tablet_bootstrap.cc:654] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:48.479714 7581 tablet_bootstrap.cc:654] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:48.479792 7580 tablet_bootstrap.cc:492] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee: No bootstrap required, opened a new log
I20260809 02:09:48.479848 7580 ts_tablet_manager.cc:1397] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 02:09:48.480014 7580 raft_consensus.cc:348] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.480111 7580 raft_consensus.cc:374] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:48.480141 7580 raft_consensus.cc:729] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821f1f626975452a92167f161383dbee, State: Initialized, Role: FOLLOWER
I20260809 02:09:48.480183 7580 consensus_queue.cc:260] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.480296 7580 ts_tablet_manager.cc:1428] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:48.480408 6954 catalog_manager.cc:4841] TS 30ff7069fc3f4a0fa8a472de7ba60094 (127.5.9.129:36689): tablet 5a55eec1ce7346dbbaf9046bf3e02410 (table test-table [id=d7ca457ef06f4b9ea995ed3818697fbe]) successfully deleted
I20260809 02:09:48.480718 7581 tablet_bootstrap.cc:492] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094: No bootstrap required, opened a new log
I20260809 02:09:48.480786 7581 ts_tablet_manager.cc:1397] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:09:48.480883 7582 tablet_bootstrap.cc:492] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d: No bootstrap required, opened a new log
I20260809 02:09:48.480934 7582 ts_tablet_manager.cc:1397] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:09:48.480935 7581 raft_consensus.cc:348] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.481010 7581 raft_consensus.cc:374] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:48.481033 7581 raft_consensus.cc:729] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ff7069fc3f4a0fa8a472de7ba60094, State: Initialized, Role: FOLLOWER
I20260809 02:09:48.481057 7582 raft_consensus.cc:348] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.481081 7581 consensus_queue.cc:260] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.481125 7582 raft_consensus.cc:374] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:48.481148 7582 raft_consensus.cc:729] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a33c8858a824442f9fd3fc9805cc318d, State: Initialized, Role: FOLLOWER
I20260809 02:09:48.481192 7581 ts_tablet_manager.cc:1428] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:48.481194 7582 consensus_queue.cc:260] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.481282 7582 ts_tablet_manager.cc:1428] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:09:48.592633 7587 raft_consensus.cc:482] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:09:48.592783 7587 raft_consensus.cc:504] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.593003 7587 leader_election.cc:288] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821f1f626975452a92167f161383dbee (127.5.9.130:36783), a33c8858a824442f9fd3fc9805cc318d (127.5.9.131:39633)
I20260809 02:09:48.593268 7280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820f9d72c45d45208ff09c408e924fa6" candidate_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821f1f626975452a92167f161383dbee" is_pre_election: true
I20260809 02:09:48.593387 7280 raft_consensus.cc:2454] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ff7069fc3f4a0fa8a472de7ba60094 in term 0.
I20260809 02:09:48.593336 7428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820f9d72c45d45208ff09c408e924fa6" candidate_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a33c8858a824442f9fd3fc9805cc318d" is_pre_election: true
I20260809 02:09:48.593478 7428 raft_consensus.cc:2454] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ff7069fc3f4a0fa8a472de7ba60094 in term 0.
I20260809 02:09:48.593601 7061 leader_election.cc:302] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ff7069fc3f4a0fa8a472de7ba60094, 821f1f626975452a92167f161383dbee; no voters:
I20260809 02:09:48.593742 7587 raft_consensus.cc:2781] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:09:48.593792 7587 raft_consensus.cc:482] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:09:48.593814 7587 raft_consensus.cc:3037] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:48.594501 7587 raft_consensus.cc:504] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.594650 7587 leader_election.cc:288] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [CANDIDATE]: Term 1 election: Requested vote from peers 821f1f626975452a92167f161383dbee (127.5.9.130:36783), a33c8858a824442f9fd3fc9805cc318d (127.5.9.131:39633)
I20260809 02:09:48.594854 7280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820f9d72c45d45208ff09c408e924fa6" candidate_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821f1f626975452a92167f161383dbee"
I20260809 02:09:48.594944 7280 raft_consensus.cc:3037] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:48.594882 7428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820f9d72c45d45208ff09c408e924fa6" candidate_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a33c8858a824442f9fd3fc9805cc318d"
I20260809 02:09:48.594986 7428 raft_consensus.cc:3037] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:48.595646 7280 raft_consensus.cc:2454] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30ff7069fc3f4a0fa8a472de7ba60094 in term 1.
I20260809 02:09:48.595722 7428 raft_consensus.cc:2454] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30ff7069fc3f4a0fa8a472de7ba60094 in term 1.
I20260809 02:09:48.595868 7061 leader_election.cc:302] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ff7069fc3f4a0fa8a472de7ba60094, 821f1f626975452a92167f161383dbee; no voters:
I20260809 02:09:48.596026 7587 raft_consensus.cc:2781] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:48.596096 7587 raft_consensus.cc:686] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [term 1 LEADER]: Becoming Leader. State: Replica: 30ff7069fc3f4a0fa8a472de7ba60094, State: Running, Role: LEADER
I20260809 02:09:48.596158 7587 consensus_queue.cc:237] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } }
I20260809 02:09:48.596871 6967 catalog_manager.cc:5495] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30ff7069fc3f4a0fa8a472de7ba60094 (127.5.9.129). New cstate: current_term: 1 leader_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ff7069fc3f4a0fa8a472de7ba60094" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 } health_report { overall_health: UNKNOWN } } }
I20260809 02:09:48.603188 7428 raft_consensus.cc:1261] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d [term 1 FOLLOWER]: Refusing update from remote peer 30ff7069fc3f4a0fa8a472de7ba60094: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:09:48.603222 7280 raft_consensus.cc:1261] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Refusing update from remote peer 30ff7069fc3f4a0fa8a472de7ba60094: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:09:48.603520 7603 consensus_queue.cc:1035] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821f1f626975452a92167f161383dbee" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 36783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:09:48.603716 7587 consensus_queue.cc:1035] T 820f9d72c45d45208ff09c408e924fa6 P 30ff7069fc3f4a0fa8a472de7ba60094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a33c8858a824442f9fd3fc9805cc318d" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:09:48.608673 7553 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 02:09:50.127568 6967 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:09:50.127691 6967 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:09:50.128708 6967 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47e744c6318b42c6826cd84617ec0e47: Sending DeleteTablet for 3 replicas of tablet 820f9d72c45d45208ff09c408e924fa6
I20260809 02:09:50.129139 7090 tablet_service.cc:1514] Processing DeleteTablet for tablet 820f9d72c45d45208ff09c408e924fa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:50 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:57110
I20260809 02:09:50.129185 7404 tablet_service.cc:1514] Processing DeleteTablet for tablet 820f9d72c45d45208ff09c408e924fa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:50 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:43310
I20260809 02:09:50.129302 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 7042
I20260809 02:09:50.129377 7259 tablet_service.cc:1514] Processing DeleteTablet for tablet 820f9d72c45d45208ff09c408e924fa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:09:50 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:52182
I20260809 02:09:50.133098 7635 tablet_replica.cc:330] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee: stopping tablet replica
I20260809 02:09:50.133229 7635 raft_consensus.cc:2229] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:09:50.133314 7635 raft_consensus.cc:2258] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:50.133725 7635 ts_tablet_manager.cc:1905] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:50.135550 7635 ts_tablet_manager.cc:1918] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:09:50.135634 7635 log.cc:1196] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/820f9d72c45d45208ff09c408e924fa6
I20260809 02:09:50.135876 7635 ts_tablet_manager.cc:1939] T 820f9d72c45d45208ff09c408e924fa6 P 821f1f626975452a92167f161383dbee: Deleting consensus metadata
I20260809 02:09:50.136320 6956 catalog_manager.cc:4841] TS 821f1f626975452a92167f161383dbee (127.5.9.130:36783): tablet 820f9d72c45d45208ff09c408e924fa6 (table test-table [id=1ad08c29f64442a99c92b137186b31dc]) successfully deleted
W20260809 02:09:50.139314 6954 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:36689 (error 108)
I20260809 02:09:50.139591 7634 tablet_replica.cc:330] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d: stopping tablet replica
I20260809 02:09:50.139689 7634 raft_consensus.cc:2229] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:09:50.139748 7634 raft_consensus.cc:2258] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:09:50.139969 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 7191
I20260809 02:09:50.141413 7634 ts_tablet_manager.cc:1905] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:09:50.142966 7637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:50.139803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:36689 (local address 127.0.0.1:57128)
0809 02:09:50.142821 (+ 3018us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:50.142920 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:36689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2985,"thread_start_us":74,"threads_started":1}
I20260809 02:09:50.143049 7634 ts_tablet_manager.cc:1918] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:09:50.143137 7634 log.cc:1196] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/820f9d72c45d45208ff09c408e924fa6
W20260809 02:09:50.143230 6954 catalog_manager.cc:4584] TS 30ff7069fc3f4a0fa8a472de7ba60094 (127.5.9.129:36689): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 820f9d72c45d45208ff09c408e924fa6: Network error: Client connection negotiation failed: client connection to 127.5.9.129:36689: connect: Connection refused (error 111)
I20260809 02:09:50.143431 7634 ts_tablet_manager.cc:1939] T 820f9d72c45d45208ff09c408e924fa6 P a33c8858a824442f9fd3fc9805cc318d: Deleting consensus metadata
I20260809 02:09:50.143849 6956 catalog_manager.cc:4841] TS a33c8858a824442f9fd3fc9805cc318d (127.5.9.131:39633): tablet 820f9d72c45d45208ff09c408e924fa6 (table test-table [id=1ad08c29f64442a99c92b137186b31dc]) successfully deleted
I20260809 02:09:50.148286 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 7338
I20260809 02:09:50.159585 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 6936
2026-08-09T02:09:50Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5848 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:09:50 dist-test-slave-p3t5 krb5kdc[7641](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:09:50 dist-test-slave-p3t5 krb5kdc[7641](info): set up 2 sockets
Aug 09 02:09:50 dist-test-slave-p3t5 krb5kdc[7641](info): commencing operation
krb5kdc: starting...
W20260809 02:09:51.262668 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:09:51 dist-test-slave-p3t5 krb5kdc[7641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:09:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:09:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:09:51.433377 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:41831
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:41831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:51.519400 7655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:51.519663 7655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:51.519690 7655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:51.520991 7655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:09:51.521049 7655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:51.521064 7655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:51.521078 7655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:09:51.521091 7655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:09:51.522763 7655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:41831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:41831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:51.523065 7655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:51.523335 7655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:09:51.528906 7655 server_base.cc:1034] running on GCE node
W20260809 02:09:51.528882 7660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:51.528872 7661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:51.528882 7663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:51.529438 7655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:51.529839 7655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:51.531024 7655 hybrid_clock.cc:648] HybridClock initialized: now 1786241391531010 us; error 49 us; skew 500 ppm
Aug 09 02:09:51 dist-test-slave-p3t5 krb5kdc[7641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241391, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:51.533717 7655 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:09:51.534276 7655 webserver.cc:460] Webserver started at http://127.5.9.190:43445/ using document root <none> and password file <none>
I20260809 02:09:51.534529 7655 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:51.534602 7655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:51.534762 7655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:51.535921 7655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "1e79fb39bba04c80a96b83bd8c8825c2"
format_stamp: "Formatted at 2026-08-09 02:09:51 on dist-test-slave-p3t5"
server_key: "b2d9bc361cf09204db7bb53ab97dde97"
server_key_iv: "a75aa0c466df553cf19e554a77ee91cf"
server_key_version: "encryptionkey@0"
I20260809 02:09:51.536345 7655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "1e79fb39bba04c80a96b83bd8c8825c2"
format_stamp: "Formatted at 2026-08-09 02:09:51 on dist-test-slave-p3t5"
server_key: "b2d9bc361cf09204db7bb53ab97dde97"
server_key_iv: "a75aa0c466df553cf19e554a77ee91cf"
server_key_version: "encryptionkey@0"
I20260809 02:09:51.538041 7655 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:09:51.538916 7670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:51.539136 7655 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:51.539258 7655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "1e79fb39bba04c80a96b83bd8c8825c2"
format_stamp: "Formatted at 2026-08-09 02:09:51 on dist-test-slave-p3t5"
server_key: "b2d9bc361cf09204db7bb53ab97dde97"
server_key_iv: "a75aa0c466df553cf19e554a77ee91cf"
server_key_version: "encryptionkey@0"
I20260809 02:09:51.539350 7655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:51.569336 7655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:51.572072 7655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:51.572265 7655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:51.576818 7655 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:41831
I20260809 02:09:51.576860 7725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:41831 every 8 connection(s)
I20260809 02:09:51.577301 7655 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:09:51.578372 7726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:51.580111 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 7655
I20260809 02:09:51.580219 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:09:51.580407 5158 external_mini_cluster.cc:1428] Setting key 98f3961c36dab82ef1519f109357f4bd
I20260809 02:09:51.581233 7726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2: Bootstrap starting.
I20260809 02:09:51.582055 7726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:51.582456 7726 log.cc:824] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:51.583407 7726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2: No bootstrap required, opened a new log
I20260809 02:09:51.585781 7726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e79fb39bba04c80a96b83bd8c8825c2" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41831 } }
I20260809 02:09:51.585937 7726 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:51.585960 7726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e79fb39bba04c80a96b83bd8c8825c2, State: Initialized, Role: FOLLOWER
I20260809 02:09:51.586104 7726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e79fb39bba04c80a96b83bd8c8825c2" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41831 } }
I20260809 02:09:51.586184 7726 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:09:51.586208 7726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:09:51.586301 7726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:51.587150 7726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e79fb39bba04c80a96b83bd8c8825c2" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41831 } }
I20260809 02:09:51.587293 7726 leader_election.cc:302] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e79fb39bba04c80a96b83bd8c8825c2; no voters:
I20260809 02:09:51.587496 7726 leader_election.cc:288] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:09:51.587572 7731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:51.587762 7731 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [term 1 LEADER]: Becoming Leader. State: Replica: 1e79fb39bba04c80a96b83bd8c8825c2, State: Running, Role: LEADER
I20260809 02:09:51.587795 7726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:09:51.587873 7731 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e79fb39bba04c80a96b83bd8c8825c2" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41831 } }
I20260809 02:09:51.588272 7733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e79fb39bba04c80a96b83bd8c8825c2. Latest consensus state: current_term: 1 leader_uuid: "1e79fb39bba04c80a96b83bd8c8825c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e79fb39bba04c80a96b83bd8c8825c2" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41831 } } }
I20260809 02:09:51.588387 7733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [sys.catalog]: This master's current role is: LEADER
I20260809 02:09:51.588591 7739 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 02:09:51 dist-test-slave-p3t5 krb5kdc[7641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:51.589159 7739 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:09:51.589488 7732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e79fb39bba04c80a96b83bd8c8825c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e79fb39bba04c80a96b83bd8c8825c2" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41831 } } }
I20260809 02:09:51.589603 7732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2 [sys.catalog]: This master's current role is: LEADER
I20260809 02:09:51.590860 7739 catalog_manager.cc:1347] Generated new cluster ID: 2c94d1807348475b8a1ab5b0946f859e
I20260809 02:09:51.590919 7739 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:09:51.593436 7729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:51.581045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38184 (local address 127.5.9.190:41831)
0809 02:09:51.581239 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:51.581249 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:51.581290 (+ 41us) server_negotiation.cc:407] Connection header received
0809 02:09:51.581949 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:51.581960 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:51.582032 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:51.582206 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:51.583258 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:51.584322 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:51.585561 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:51.586165 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:51.586306 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:51.589656 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:51.589661 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:51.589661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:51.589675 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:51.591675 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:51.592130 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:51.592133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:51.592134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:51.592236 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:51.592483 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:51.592483 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:51.592484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:51.592744 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:51.592916 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:51.593061 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 02:09:51.593216 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
I20260809 02:09:51.602234 7739 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:09:51.602826 7739 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:09:51.610251 7739 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e79fb39bba04c80a96b83bd8c8825c2: Generated new TSK 0
I20260809 02:09:51.610503 7739 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:09:51.645269 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41831
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:51.733004 7753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:51.733301 7753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:51.733348 7753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:51.734692 7753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:51.734768 7753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:51.734815 7753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:09:51.736418 7753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:51.736727 7753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:51.737007 7753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:51.742511 7762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:51.742511 7759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:51.742516 7760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:51.742967 7753 server_base.cc:1034] running on GCE node
I20260809 02:09:51.743233 7753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:51.743515 7753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:51.744756 7753 hybrid_clock.cc:648] HybridClock initialized: now 1786241391744744 us; error 31 us; skew 500 ppm
Aug 09 02:09:51 dist-test-slave-p3t5 krb5kdc[7641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241391, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:51.747666 7753 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:09:51.748160 7753 webserver.cc:460] Webserver started at http://127.5.9.129:38675/ using document root <none> and password file <none>
I20260809 02:09:51.748399 7753 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:51.748456 7753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:51.748557 7753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:51.749711 7753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "3433177b7fd841aeb15419a96d8d9b88"
format_stamp: "Formatted at 2026-08-09 02:09:51 on dist-test-slave-p3t5"
server_key: "d6550dd6fc02cbf29dddd19e525373d8"
server_key_iv: "54a2a206c2fa318206d4edc4b04a9de0"
server_key_version: "encryptionkey@0"
I20260809 02:09:51.750065 7753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "3433177b7fd841aeb15419a96d8d9b88"
format_stamp: "Formatted at 2026-08-09 02:09:51 on dist-test-slave-p3t5"
server_key: "d6550dd6fc02cbf29dddd19e525373d8"
server_key_iv: "54a2a206c2fa318206d4edc4b04a9de0"
server_key_version: "encryptionkey@0"
I20260809 02:09:51.751508 7753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:09:51.752295 7770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:51.752545 7753 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:09:51.752648 7753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "3433177b7fd841aeb15419a96d8d9b88"
format_stamp: "Formatted at 2026-08-09 02:09:51 on dist-test-slave-p3t5"
server_key: "d6550dd6fc02cbf29dddd19e525373d8"
server_key_iv: "54a2a206c2fa318206d4edc4b04a9de0"
server_key_version: "encryptionkey@0"
I20260809 02:09:51.752741 7753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:51.783342 7753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:51.786091 7753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:51.786263 7753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:51.786552 7753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:09:51.786943 7753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:51.787003 7753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:51.787052 7753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:51.787081 7753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:51.793587 7753 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:40667
I20260809 02:09:51.793692 7893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:40667 every 8 connection(s)
I20260809 02:09:51.794085 7753 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:09:51 dist-test-slave-p3t5 krb5kdc[7641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241391, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:51.802552 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 7753
I20260809 02:09:51.802680 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:09:51.802868 5158 external_mini_cluster.cc:1428] Setting key fc7f27fcd628e1d8b7f7fbb4787959f2
I20260809 02:09:51.804905 7729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:51.795041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:54367 (local address 127.5.9.190:41831)
0809 02:09:51.795111 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:51.795113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:51.795350 (+ 237us) server_negotiation.cc:407] Connection header received
0809 02:09:51.795848 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:51.795849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:51.795860 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:51.795926 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:51.796975 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:51.797509 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:51.798671 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:51.798877 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:51.798938 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:51.801490 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:51.801492 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:51.801492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:51.801500 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:51.803256 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:51.803789 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:51.803790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:51.803791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:51.803862 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:51.804133 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:51.804134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:51.804135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:51.804297 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:51.804368 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:51.804646 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 02:09:51.804761 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:09:51.804929 7896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:51.794984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:41831 (local address 127.5.9.129:54367)
0809 02:09:51.795166 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:51.795215 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:51.795737 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:51.796016 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:51.796020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:51.796658 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:51.796865 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:51.796870 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:51.797604 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:51.797605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:51.798581 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:51.798584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:51.798989 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:51.798991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:51.799119 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:51.799776 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:51.799786 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:51.801386 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:51.803342 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:51.803353 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:51.803356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:51.803706 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:51.803935 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:51.803936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:51.803936 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:51.804062 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:51.804364 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:51.804369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:51.804511 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 02:09:51.804600 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 02:09:51.804741 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260809 02:09:51.805528 7894 heartbeater.cc:347] Connected to a master server at 127.5.9.190:41831
I20260809 02:09:51.805673 7894 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:51.805922 7894 heartbeater.cc:511] Master 127.5.9.190:41831 requested a full tablet report, sending...
I20260809 02:09:51.806591 7689 ts_manager.cc:194] Registered new tserver with Master: 3433177b7fd841aeb15419a96d8d9b88 (127.5.9.129:40667)
I20260809 02:09:51.807560 7689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:54367
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:09:51.833949 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41831
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:51.918506 7899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:51.918762 7899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:51.918789 7899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:51.920428 7899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:51.920500 7899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:51.920533 7899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:09:51.922184 7899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:51.922418 7899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:51.922622 7899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:51.928254 7905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:51.928232 7904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:51.928432 7899 server_base.cc:1034] running on GCE node
W20260809 02:09:51.928279 7907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:51.928725 7899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:51.929008 7899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:51.930203 7899 hybrid_clock.cc:648] HybridClock initialized: now 1786241391930188 us; error 46 us; skew 500 ppm
Aug 09 02:09:51 dist-test-slave-p3t5 krb5kdc[7641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241391, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:51.932969 7899 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:09:51.933480 7899 webserver.cc:460] Webserver started at http://127.5.9.130:35595/ using document root <none> and password file <none>
I20260809 02:09:51.933772 7899 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:51.933869 7899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:51.934031 7899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:51.935223 7899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "154b04b5ac46455f8cae380776fb0cf0"
format_stamp: "Formatted at 2026-08-09 02:09:51 on dist-test-slave-p3t5"
server_key: "ea90633fd2b6dac5ac28f073c0c9667a"
server_key_iv: "b9ef748efdce814ab5906da0b3e43f5c"
server_key_version: "encryptionkey@0"
I20260809 02:09:51.935624 7899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "154b04b5ac46455f8cae380776fb0cf0"
format_stamp: "Formatted at 2026-08-09 02:09:51 on dist-test-slave-p3t5"
server_key: "ea90633fd2b6dac5ac28f073c0c9667a"
server_key_iv: "b9ef748efdce814ab5906da0b3e43f5c"
server_key_version: "encryptionkey@0"
I20260809 02:09:51.937062 7899 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:09:51.938023 7914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:51.938256 7899 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:51.938351 7899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "154b04b5ac46455f8cae380776fb0cf0"
format_stamp: "Formatted at 2026-08-09 02:09:51 on dist-test-slave-p3t5"
server_key: "ea90633fd2b6dac5ac28f073c0c9667a"
server_key_iv: "b9ef748efdce814ab5906da0b3e43f5c"
server_key_version: "encryptionkey@0"
I20260809 02:09:51.938446 7899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:51.951874 7899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:51.954224 7899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:51.954351 7899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:51.954564 7899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:09:51.954921 7899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:51.954980 7899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:51.955027 7899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:51.955054 7899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:51.961189 7899 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:41049
I20260809 02:09:51.961274 8036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:41049 every 8 connection(s)
I20260809 02:09:51.961697 7899 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:09:51 dist-test-slave-p3t5 krb5kdc[7641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241391, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:51.970470 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 7899
I20260809 02:09:51.970628 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:09:51.970854 5158 external_mini_cluster.cc:1428] Setting key c0ba4915f89cf0ef8602da59eae34c50
I20260809 02:09:51.972234 7729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:51.962573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:50331 (local address 127.5.9.190:41831)
0809 02:09:51.962624 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:51.962626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:51.962892 (+ 266us) server_negotiation.cc:407] Connection header received
0809 02:09:51.963444 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:51.963446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:51.963459 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:51.963518 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:51.964561 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:51.965083 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:51.966228 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:51.966436 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:51.966521 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:51.968990 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:51.968992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:51.968992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:51.969000 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:51.970455 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:51.971056 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:51.971057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:51.971057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:51.971135 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:51.971443 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:51.971444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:51.971444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:51.971602 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:51.971676 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:51.972087 (+ 411us) server_negotiation.cc:299] Negotiation successful
0809 02:09:51.972164 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:09:51.972563 8039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:51.962549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:41831 (local address 127.5.9.130:50331)
0809 02:09:51.962750 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:51.962790 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:51.963330 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:51.963605 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:51.963609 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:51.964235 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:51.964449 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:51.964455 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:51.965190 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:51.965191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:51.966134 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:51.966137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:51.966534 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:51.966535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:51.966662 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:51.967260 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:51.967270 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:51.968889 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:51.970548 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:51.970557 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:51.970560 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:51.970969 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:51.971208 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:51.971209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:51.971209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:51.971367 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:51.971689 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:51.971697 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:51.971929 (+ 232us) client_negotiation.cc:769] Sending connection context
0809 02:09:51.972059 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 02:09:51.972256 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
I20260809 02:09:51.973136 8037 heartbeater.cc:347] Connected to a master server at 127.5.9.190:41831
I20260809 02:09:51.973269 8037 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:51.973483 8037 heartbeater.cc:511] Master 127.5.9.190:41831 requested a full tablet report, sending...
I20260809 02:09:51.973944 7689 ts_manager.cc:194] Registered new tserver with Master: 154b04b5ac46455f8cae380776fb0cf0 (127.5.9.130:41049)
I20260809 02:09:51.974354 7689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:50331
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:09:52.001318 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41831
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:52.089145 8042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:52.089398 8042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:52.089447 8042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:52.090919 8042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:52.090984 8042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:52.091015 8042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:09:52.092638 8042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:52.092945 8042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:52.093223 8042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:52.098654 8050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:52.098670 8047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:52.098647 8048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:52.099030 8042 server_base.cc:1034] running on GCE node
I20260809 02:09:52.099258 8042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:52.099531 8042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:52.100701 8042 hybrid_clock.cc:648] HybridClock initialized: now 1786241392100697 us; error 39 us; skew 500 ppm
Aug 09 02:09:52 dist-test-slave-p3t5 krb5kdc[7641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241392, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:52.103477 8042 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:09:52.104003 8042 webserver.cc:460] Webserver started at http://127.5.9.131:46187/ using document root <none> and password file <none>
I20260809 02:09:52.104249 8042 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:52.104321 8042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:52.104480 8042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:52.105666 8042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "2a01a8cd294f42a0a336847981a31d3b"
format_stamp: "Formatted at 2026-08-09 02:09:52 on dist-test-slave-p3t5"
server_key: "122641accc5c3b011a48a958c5054656"
server_key_iv: "e803ad4013168e5154ebcc7162878268"
server_key_version: "encryptionkey@0"
I20260809 02:09:52.106082 8042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "2a01a8cd294f42a0a336847981a31d3b"
format_stamp: "Formatted at 2026-08-09 02:09:52 on dist-test-slave-p3t5"
server_key: "122641accc5c3b011a48a958c5054656"
server_key_iv: "e803ad4013168e5154ebcc7162878268"
server_key_version: "encryptionkey@0"
I20260809 02:09:52.107764 8042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:09:52.108588 8057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:52.108774 8042 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:52.108884 8042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "2a01a8cd294f42a0a336847981a31d3b"
format_stamp: "Formatted at 2026-08-09 02:09:52 on dist-test-slave-p3t5"
server_key: "122641accc5c3b011a48a958c5054656"
server_key_iv: "e803ad4013168e5154ebcc7162878268"
server_key_version: "encryptionkey@0"
I20260809 02:09:52.108969 8042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:52.129030 8042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:52.131722 8042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:52.131922 8042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:52.132202 8042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:09:52.132552 8042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:52.132607 8042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:52.132642 8042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:52.132669 8042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:52.139109 8042 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:41247
I20260809 02:09:52.139155 8177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:41247 every 8 connection(s)
I20260809 02:09:52.139524 8042 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
Aug 09 02:09:52 dist-test-slave-p3t5 krb5kdc[7641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241392, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:52.148350 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 8042
I20260809 02:09:52.148469 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:09:52.148697 5158 external_mini_cluster.cc:1428] Setting key 380c6b86e676112b30628372ef2f6c7c
I20260809 02:09:52.150067 7729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:52.140314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:39299 (local address 127.5.9.190:41831)
0809 02:09:52.140418 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:52.140420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:52.140605 (+ 185us) server_negotiation.cc:407] Connection header received
0809 02:09:52.141158 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:52.141159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:52.141171 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:52.141228 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:52.142503 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:52.143048 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:52.144174 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:52.144392 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:52.144464 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:52.146860 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:52.146862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:52.146862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:52.146871 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:52.148329 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:52.148954 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:52.148955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:52.148955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:52.149030 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:52.149321 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:52.149322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:52.149323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:52.149458 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:52.149541 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:52.149870 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 02:09:52.149970 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:09:52.150282 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:52.140271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:41831 (local address 127.5.9.131:39299)
0809 02:09:52.140473 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:52.140513 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:52.141022 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:52.141334 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:52.141339 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:52.142078 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:52.142357 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:52.142365 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:52.143137 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:52.143138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:52.144071 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:52.144075 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:52.144491 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:52.144492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:52.144603 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:52.145149 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:52.145159 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:52.146761 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:52.148420 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:52.148452 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:52.148455 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:52.148846 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:52.149122 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:52.149123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:52.149123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:52.149251 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:52.149530 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:52.149537 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:52.149734 (+ 197us) client_negotiation.cc:769] Sending connection context
0809 02:09:52.149850 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 02:09:52.150029 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":108,"threads_started":1}
I20260809 02:09:52.150776 8178 heartbeater.cc:347] Connected to a master server at 127.5.9.190:41831
I20260809 02:09:52.150902 8178 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:52.151106 8178 heartbeater.cc:511] Master 127.5.9.190:41831 requested a full tablet report, sending...
I20260809 02:09:52.151485 7689 ts_manager.cc:194] Registered new tserver with Master: 2a01a8cd294f42a0a336847981a31d3b (127.5.9.131:41247)
I20260809 02:09:52.151871 7689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:39299
I20260809 02:09:52.160689 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 02:09:52.174891 7729 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:09:52.173741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38186 (local address 127.5.9.190:41831)
0809 02:09:52.173792 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:52.173799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:52.173994 (+ 195us) server_negotiation.cc:407] Connection header received
0809 02:09:52.174075 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:52.174076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:52.174088 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:52.174161 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:52.174791 (+ 630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38186: BlockingRecv error: recv got EOF from 127.0.0.1:38186 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:09:52.175854 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 7753
I20260809 02:09:52.182179 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 7899
I20260809 02:09:52.188169 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 8042
I20260809 02:09:52.195158 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 7655
2026-08-09T02:09:52Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2033 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:09:52 dist-test-slave-p3t5 krb5kdc[8190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:09:52 dist-test-slave-p3t5 krb5kdc[8190](info): set up 2 sockets
Aug 09 02:09:52 dist-test-slave-p3t5 krb5kdc[8190](info): commencing operation
krb5kdc: starting...
W20260809 02:09:53.316946 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260809 02:09:53.544821 5158 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.228s user 0.000s sys 0.003s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260809 02:09:53.796257 5158 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.251s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260809 02:09:53.975147 5158 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.179s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/krb5kdc/test-user.keytab.
W20260809 02:09:54.629225 5158 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.654s user 0.000s sys 0.002s
Aug 09 02:09:54 dist-test-slave-p3t5 krb5kdc[8190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:09:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:09:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
W20260809 02:09:54.757364 5158 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.5.9.190: real 0.101s user 0.000s sys 0.004s
I20260809 02:09:54.757879 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:34759
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:35877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:34759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:54.847489 8204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:54.847733 8204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:54.847759 8204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:54.849009 8204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:09:54.849053 8204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:54.849068 8204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:54.849081 8204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:09:54.849093 8204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:09:54.850817 8204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:35877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:34759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:34759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:54.851071 8204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:54.851267 8204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:54.856969 8211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:54.856969 8213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:54.856993 8210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:54.857175 8204 server_base.cc:1034] running on GCE node
I20260809 02:09:54.857577 8204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:54.857925 8204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:54.859124 8204 hybrid_clock.cc:648] HybridClock initialized: now 1786241394859094 us; error 53 us; skew 500 ppm
Aug 09 02:09:54 dist-test-slave-p3t5 krb5kdc[8190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241394, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:54.867668 8204 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:09:54.868207 8204 webserver.cc:460] Webserver started at http://127.5.9.190:44407/ using document root <none> and password file <none>
I20260809 02:09:54.868487 8204 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:54.868563 8204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:54.868732 8204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:54.870167 8204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "dc7d4d483f5746acb1bb8cd3767e45d7"
format_stamp: "Formatted at 2026-08-09 02:09:54 on dist-test-slave-p3t5"
server_key: "1dc227e34996c5025bdc0a794efc4129"
server_key_iv: "53e6b3a813b4a42a50879125ead77473"
server_key_version: "encryptionkey@0"
I20260809 02:09:54.870751 8204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "dc7d4d483f5746acb1bb8cd3767e45d7"
format_stamp: "Formatted at 2026-08-09 02:09:54 on dist-test-slave-p3t5"
server_key: "1dc227e34996c5025bdc0a794efc4129"
server_key_iv: "53e6b3a813b4a42a50879125ead77473"
server_key_version: "encryptionkey@0"
I20260809 02:09:54.872325 8204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:09:54.873142 8221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:54.873333 8204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:54.873441 8204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "dc7d4d483f5746acb1bb8cd3767e45d7"
format_stamp: "Formatted at 2026-08-09 02:09:54 on dist-test-slave-p3t5"
server_key: "1dc227e34996c5025bdc0a794efc4129"
server_key_iv: "53e6b3a813b4a42a50879125ead77473"
server_key_version: "encryptionkey@0"
I20260809 02:09:54.873533 8204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:54.901980 8204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:54.904381 8204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:54.904582 8204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:54.908994 8204 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:34759
I20260809 02:09:54.909077 8276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:34759 every 8 connection(s)
I20260809 02:09:54.909459 8204 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:09:54.910485 8277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:54.913000 8277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7: Bootstrap starting.
I20260809 02:09:54.913774 8277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:54.914124 8277 log.cc:824] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:54.914919 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 8204
I20260809 02:09:54.915027 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:09:54.915100 8277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7: No bootstrap required, opened a new log
I20260809 02:09:54.915230 5158 external_mini_cluster.cc:1428] Setting key 37e80dc963bcef2871f6205364d66b03
I20260809 02:09:54.918303 8277 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc7d4d483f5746acb1bb8cd3767e45d7" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34759 } }
I20260809 02:09:54.918469 8277 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:54.918494 8277 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc7d4d483f5746acb1bb8cd3767e45d7, State: Initialized, Role: FOLLOWER
I20260809 02:09:54.918643 8277 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc7d4d483f5746acb1bb8cd3767e45d7" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34759 } }
I20260809 02:09:54.918726 8277 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:09:54.918771 8277 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:09:54.918823 8277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:54.919453 8277 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc7d4d483f5746acb1bb8cd3767e45d7" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34759 } }
I20260809 02:09:54.919548 8277 leader_election.cc:302] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc7d4d483f5746acb1bb8cd3767e45d7; no voters:
I20260809 02:09:54.919778 8277 leader_election.cc:288] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:09:54.919811 8283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:54.919932 8283 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [term 1 LEADER]: Becoming Leader. State: Replica: dc7d4d483f5746acb1bb8cd3767e45d7, State: Running, Role: LEADER
I20260809 02:09:54.920022 8283 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc7d4d483f5746acb1bb8cd3767e45d7" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34759 } }
I20260809 02:09:54.920089 8277 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:09:54.920360 8283 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc7d4d483f5746acb1bb8cd3767e45d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc7d4d483f5746acb1bb8cd3767e45d7" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34759 } } }
I20260809 02:09:54.920542 8283 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [sys.catalog]: This master's current role is: LEADER
I20260809 02:09:54.920751 8283 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc7d4d483f5746acb1bb8cd3767e45d7. Latest consensus state: current_term: 1 leader_uuid: "dc7d4d483f5746acb1bb8cd3767e45d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc7d4d483f5746acb1bb8cd3767e45d7" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34759 } } }
I20260809 02:09:54.920890 8283 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7 [sys.catalog]: This master's current role is: LEADER
I20260809 02:09:54.921198 8289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:09:54.921931 8289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 02:09:54 dist-test-slave-p3t5 krb5kdc[8190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:54.923934 8289 catalog_manager.cc:1347] Generated new cluster ID: f19aff51f5ce4073a9c8cec37639a462
I20260809 02:09:54.923993 8289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:09:54.927718 8282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:54.915808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46212 (local address 127.5.9.190:34759)
0809 02:09:54.916023 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:54.916032 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:54.916062 (+ 30us) server_negotiation.cc:407] Connection header received
0809 02:09:54.916611 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:54.916619 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:54.916672 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:54.916796 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:54.917724 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:54.918830 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:54.919687 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:54.920400 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:54.920898 (+ 498us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:54.923673 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:54.923678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:54.923678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:54.923691 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:54.925813 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:54.926275 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:54.926279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:54.926279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:54.926409 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:54.926696 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:54.926697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:54.926697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:54.927037 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:54.927201 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:54.927348 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 02:09:54.927494 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":114,"threads_started":1}
I20260809 02:09:54.928889 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:09:54.933844 8289 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:09:54.934638 8289 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:09:54.942363 8289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc7d4d483f5746acb1bb8cd3767e45d7: Generated new TSK 0
I20260809 02:09:54.942624 8289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:09:54.947444 8282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:54.943217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46218 (local address 127.5.9.190:34759)
0809 02:09:54.943264 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:54.943266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:54.943385 (+ 119us) server_negotiation.cc:407] Connection header received
0809 02:09:54.943473 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:54.943474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:54.943483 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:54.943530 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:54.944058 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:54.944620 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:54.945843 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:54.946058 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:54.946120 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:54.946912 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:54.946913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:54.946913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:09:54.947120 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:54.947141 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:09:54.947147 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:54.947196 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:54.947301 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 02:09:54.947331 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:09:54.950750 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 8204
2026-08-09T02:09:54Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2753 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:09:55 dist-test-slave-p3t5 krb5kdc[8312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:09:55 dist-test-slave-p3t5 krb5kdc[8312](info): set up 2 sockets
Aug 09 02:09:55 dist-test-slave-p3t5 krb5kdc[8312](info): commencing operation
krb5kdc: starting...
W20260809 02:09:56.022532 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:09:56 dist-test-slave-p3t5 krb5kdc[8312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:09:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:09:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:09:56.160509 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:37441
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:37441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:56.246953 8326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:56.247231 8326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:56.247274 8326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:56.248493 8326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:09:56.248571 8326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:56.248595 8326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:56.248616 8326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:09:56.248646 8326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:09:56.250247 8326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:37441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:37441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:56.250550 8326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:56.250813 8326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:56.256150 8333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:56.256150 8334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:56.256178 8336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:56.256491 8326 server_base.cc:1034] running on GCE node
I20260809 02:09:56.256764 8326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:56.257090 8326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:56.258268 8326 hybrid_clock.cc:648] HybridClock initialized: now 1786241396258253 us; error 33 us; skew 500 ppm
Aug 09 02:09:56 dist-test-slave-p3t5 krb5kdc[8312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241396, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:56.261041 8326 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:09:56.261560 8326 webserver.cc:460] Webserver started at http://127.5.9.190:44915/ using document root <none> and password file <none>
I20260809 02:09:56.261859 8326 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:56.261909 8326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:56.262009 8326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:56.263116 8326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "949e885ea38c42b48f454c56362fb512"
format_stamp: "Formatted at 2026-08-09 02:09:56 on dist-test-slave-p3t5"
server_key: "140fdb526dc3df0aa13197b45d2ed542"
server_key_iv: "db6ab278298e31952e97e85dae27b134"
server_key_version: "encryptionkey@0"
I20260809 02:09:56.263438 8326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "949e885ea38c42b48f454c56362fb512"
format_stamp: "Formatted at 2026-08-09 02:09:56 on dist-test-slave-p3t5"
server_key: "140fdb526dc3df0aa13197b45d2ed542"
server_key_iv: "db6ab278298e31952e97e85dae27b134"
server_key_version: "encryptionkey@0"
I20260809 02:09:56.264873 8326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:09:56.265817 8343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:56.266031 8326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:56.266129 8326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "949e885ea38c42b48f454c56362fb512"
format_stamp: "Formatted at 2026-08-09 02:09:56 on dist-test-slave-p3t5"
server_key: "140fdb526dc3df0aa13197b45d2ed542"
server_key_iv: "db6ab278298e31952e97e85dae27b134"
server_key_version: "encryptionkey@0"
I20260809 02:09:56.266194 8326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:56.288658 8326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:56.291690 8326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:56.291922 8326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:56.296277 8326 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:37441
I20260809 02:09:56.296319 8402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:37441 every 8 connection(s)
I20260809 02:09:56.296726 8326 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:09:56.297199 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 8326
I20260809 02:09:56.297328 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:09:56.297617 5158 external_mini_cluster.cc:1428] Setting key 3e25f17847e9f5208b1bbd9e7704ff68
I20260809 02:09:56.298357 8403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:56.301584 8403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512: Bootstrap starting.
I20260809 02:09:56.302356 8403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:56.302675 8403 log.cc:824] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:56.303553 8403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512: No bootstrap required, opened a new log
Aug 09 02:09:56 dist-test-slave-p3t5 krb5kdc[8312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:56.305883 8403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949e885ea38c42b48f454c56362fb512" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37441 } }
I20260809 02:09:56.306094 8403 raft_consensus.cc:374] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:56.306135 8403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 949e885ea38c42b48f454c56362fb512, State: Initialized, Role: FOLLOWER
I20260809 02:09:56.306277 8403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949e885ea38c42b48f454c56362fb512" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37441 } }
I20260809 02:09:56.306370 8403 raft_consensus.cc:388] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:09:56.306393 8403 raft_consensus.cc:482] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:09:56.306469 8403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:56.307255 8403 raft_consensus.cc:504] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949e885ea38c42b48f454c56362fb512" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37441 } }
I20260809 02:09:56.307358 8403 leader_election.cc:302] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 949e885ea38c42b48f454c56362fb512; no voters:
I20260809 02:09:56.307610 8403 leader_election.cc:288] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:09:56.307648 8408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:56.307772 8408 raft_consensus.cc:686] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [term 1 LEADER]: Becoming Leader. State: Replica: 949e885ea38c42b48f454c56362fb512, State: Running, Role: LEADER
I20260809 02:09:56.307876 8408 consensus_queue.cc:237] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949e885ea38c42b48f454c56362fb512" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37441 } }
I20260809 02:09:56.307915 8403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:09:56.308264 8408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 949e885ea38c42b48f454c56362fb512. Latest consensus state: current_term: 1 leader_uuid: "949e885ea38c42b48f454c56362fb512" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949e885ea38c42b48f454c56362fb512" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37441 } } }
I20260809 02:09:56.308341 8408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [sys.catalog]: This master's current role is: LEADER
I20260809 02:09:56.308296 8410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "949e885ea38c42b48f454c56362fb512" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "949e885ea38c42b48f454c56362fb512" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37441 } } }
I20260809 02:09:56.308454 8410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512 [sys.catalog]: This master's current role is: LEADER
I20260809 02:09:56.309866 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:56.298281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32968 (local address 127.5.9.190:37441)
0809 02:09:56.298459 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:56.298470 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:56.298504 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:09:56.299002 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:56.299010 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:56.299061 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:56.299181 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:56.300027 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:56.300985 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:56.302120 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:56.302616 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:56.302710 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:56.305675 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:56.305680 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:56.305680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:56.305692 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:56.307893 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:56.308367 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:56.308370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:56.308371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:56.308489 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:56.308733 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:56.308734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:56.308734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:56.309068 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:56.309330 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:56.309408 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 02:09:56.309562 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260809 02:09:56.310575 8414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:09:56.311240 8414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:09:56.312891 8414 catalog_manager.cc:1347] Generated new cluster ID: 26ceee83ed89450e9b56811d15300b37
I20260809 02:09:56.312954 8414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:09:56.326292 8414 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:09:56.327138 8414 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:09:56.332298 8414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 949e885ea38c42b48f454c56362fb512: Generated new TSK 0
I20260809 02:09:56.332540 8414 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:09:56.365293 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:37441
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:56.453814 8431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:56.454047 8431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:56.454074 8431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:56.455274 8431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:56.455327 8431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:56.455355 8431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:09:56.456882 8431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:37441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:56.457109 8431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:56.457304 8431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:56.462893 8436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:56.462886 8439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:56.462888 8437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:56.463068 8431 server_base.cc:1034] running on GCE node
I20260809 02:09:56.463279 8431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:56.463506 8431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:56.464655 8431 hybrid_clock.cc:648] HybridClock initialized: now 1786241396464637 us; error 30 us; skew 500 ppm
Aug 09 02:09:56 dist-test-slave-p3t5 krb5kdc[8312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241396, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:56.467542 8431 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:09:56.468052 8431 webserver.cc:460] Webserver started at http://127.5.9.129:41787/ using document root <none> and password file <none>
I20260809 02:09:56.468361 8431 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:56.468451 8431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:56.468624 8431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:56.469956 8431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "85807e14f1ac4268b961f50a9ff18764"
format_stamp: "Formatted at 2026-08-09 02:09:56 on dist-test-slave-p3t5"
server_key: "8e274856537b6e2a62999bd08551ea43"
server_key_iv: "9bf808c585168f5ce1bcd9d69129aa5f"
server_key_version: "encryptionkey@0"
I20260809 02:09:56.470357 8431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "85807e14f1ac4268b961f50a9ff18764"
format_stamp: "Formatted at 2026-08-09 02:09:56 on dist-test-slave-p3t5"
server_key: "8e274856537b6e2a62999bd08551ea43"
server_key_iv: "9bf808c585168f5ce1bcd9d69129aa5f"
server_key_version: "encryptionkey@0"
I20260809 02:09:56.471858 8431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:09:56.472649 8446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:56.472851 8431 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:56.472958 8431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "85807e14f1ac4268b961f50a9ff18764"
format_stamp: "Formatted at 2026-08-09 02:09:56 on dist-test-slave-p3t5"
server_key: "8e274856537b6e2a62999bd08551ea43"
server_key_iv: "9bf808c585168f5ce1bcd9d69129aa5f"
server_key_version: "encryptionkey@0"
I20260809 02:09:56.473050 8431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:56.480995 8431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:56.483620 8431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:56.483805 8431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:56.484078 8431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:09:56.484416 8431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:56.484473 8431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:56.484520 8431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:56.484545 8431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:56.490809 8431 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:36795
I20260809 02:09:56.490844 8563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:36795 every 8 connection(s)
I20260809 02:09:56.491209 8431 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:09:56.491628 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 8431
I20260809 02:09:56.491776 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:09:56.492070 5158 external_mini_cluster.cc:1428] Setting key a40d627c7951440048b3b1faaf7bc069
Aug 09 02:09:56 dist-test-slave-p3t5 krb5kdc[8312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241396, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
I20260809 02:09:56.502983 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:56.492069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:41577 (local address 127.5.9.190:37441)
0809 02:09:56.492152 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:56.492154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:56.492434 (+ 280us) server_negotiation.cc:407] Connection header received
0809 02:09:56.493263 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:56.493264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:56.493274 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:56.493329 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:56.494940 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:56.495572 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:56.496726 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:56.496986 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:56.497085 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:56.499590 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:56.499591 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:56.499592 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:56.499601 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:56.501177 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:56.501750 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:56.501752 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:56.501752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:56.501830 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:56.502142 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:56.502143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:56.502143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:56.502290 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:56.502372 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:56.502791 (+ 419us) server_negotiation.cc:299] Negotiation successful
0809 02:09:56.502909 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:09:56.503151 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:56.492036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:37441 (local address 127.5.9.129:41577)
0809 02:09:56.492254 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:56.492317 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:56.493124 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:56.493413 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:56.493418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:56.494441 (+ 1023us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:56.494792 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:56.494800 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:56.495682 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:56.495682 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:56.496611 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:56.496614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:56.497098 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:56.497099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:56.497246 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:56.497862 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:56.497872 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:56.499470 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:56.501295 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:56.501305 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:56.501308 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:56.501653 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:56.501908 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:56.501909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:56.501910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:56.502052 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:56.502373 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:56.502382 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:56.502608 (+ 226us) client_negotiation.cc:769] Sending connection context
0809 02:09:56.502733 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 02:09:56.502909 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":101,"threads_started":1}
I20260809 02:09:56.503726 8564 heartbeater.cc:347] Connected to a master server at 127.5.9.190:37441
I20260809 02:09:56.503887 8564 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:56.504098 8564 heartbeater.cc:511] Master 127.5.9.190:37441 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
I20260809 02:09:56.504772 8365 ts_manager.cc:194] Registered new tserver with Master: 85807e14f1ac4268b961f50a9ff18764 (127.5.9.129:36795)
I20260809 02:09:56.505468 8365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:41577
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:09:56.523030 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:37441
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:56.612753 8570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:56.612993 8570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:56.613020 8570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:56.614251 8570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:56.614295 8570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:56.614322 8570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:09:56.615811 8570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:37441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:56.616043 8570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:56.616245 8570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:56.621547 8576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:56.621733 8570 server_base.cc:1034] running on GCE node
W20260809 02:09:56.621560 8575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:56.621634 8578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:56.622053 8570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:56.622328 8570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:56.623515 8570 hybrid_clock.cc:648] HybridClock initialized: now 1786241396623501 us; error 47 us; skew 500 ppm
Aug 09 02:09:56 dist-test-slave-p3t5 krb5kdc[8312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241396, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:56.626430 8570 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:09:56.626976 8570 webserver.cc:460] Webserver started at http://127.5.9.130:41679/ using document root <none> and password file <none>
I20260809 02:09:56.627249 8570 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:56.627326 8570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:56.627486 8570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:56.628608 8570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "39398a776dc64ce1a8d3753a1e3ab8d6"
format_stamp: "Formatted at 2026-08-09 02:09:56 on dist-test-slave-p3t5"
server_key: "9dac50fca7cd7dc317ad98eea2f07c21"
server_key_iv: "85265d3095adb2d9f034ca3a4c7808f3"
server_key_version: "encryptionkey@0"
I20260809 02:09:56.628998 8570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "39398a776dc64ce1a8d3753a1e3ab8d6"
format_stamp: "Formatted at 2026-08-09 02:09:56 on dist-test-slave-p3t5"
server_key: "9dac50fca7cd7dc317ad98eea2f07c21"
server_key_iv: "85265d3095adb2d9f034ca3a4c7808f3"
server_key_version: "encryptionkey@0"
I20260809 02:09:56.630539 8570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:09:56.631378 8586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:56.631561 8570 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:56.631672 8570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "39398a776dc64ce1a8d3753a1e3ab8d6"
format_stamp: "Formatted at 2026-08-09 02:09:56 on dist-test-slave-p3t5"
server_key: "9dac50fca7cd7dc317ad98eea2f07c21"
server_key_iv: "85265d3095adb2d9f034ca3a4c7808f3"
server_key_version: "encryptionkey@0"
I20260809 02:09:56.631754 8570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:56.657562 8570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:56.660224 8570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:56.660408 8570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:56.660671 8570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:09:56.661010 8570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:56.661067 8570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:56.661114 8570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:56.661164 8570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:56.667816 8570 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:36273
I20260809 02:09:56.667886 8702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:36273 every 8 connection(s)
I20260809 02:09:56.668242 8570 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:09:56.669734 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 8570
I20260809 02:09:56.669916 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:09:56.670136 5158 external_mini_cluster.cc:1428] Setting key b7867ad68de757e93d87b2c488da560b
Aug 09 02:09:56 dist-test-slave-p3t5 krb5kdc[8312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241396, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:56.679862 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:56.669117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:48967 (local address 127.5.9.190:37441)
0809 02:09:56.669167 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:56.669169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:56.669416 (+ 247us) server_negotiation.cc:407] Connection header received
0809 02:09:56.670120 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:56.670121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:56.670135 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:56.670203 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:56.671435 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:56.672001 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:56.673198 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:56.673518 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:56.673645 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:56.676406 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:56.676408 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:56.676408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:56.676419 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:56.678276 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:56.678751 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:56.678752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:56.678753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:56.678828 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:56.679088 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:56.679089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:56.679089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:56.679261 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:56.679343 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:56.679639 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 02:09:56.679739 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 02:09:56.680018 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:56.669077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:37441 (local address 127.5.9.130:48967)
0809 02:09:56.669286 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:56.669326 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:56.670011 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:56.670371 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:56.670376 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:56.671109 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:56.671324 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:56.671329 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:56.672106 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:56.672107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:56.673100 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:56.673103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:56.673719 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:56.673723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:56.673867 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:56.674505 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:56.674514 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:56.676282 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:56.678390 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:56.678399 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:56.678402 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:56.678663 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:56.678903 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:56.678904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:56.678904 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:56.679018 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:56.679334 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:56.679338 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:56.679488 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 02:09:56.679578 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 02:09:56.679723 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
I20260809 02:09:56.680624 8703 heartbeater.cc:347] Connected to a master server at 127.5.9.190:37441
I20260809 02:09:56.680748 8703 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:56.680953 8703 heartbeater.cc:511] Master 127.5.9.190:37441 requested a full tablet report, sending...
I20260809 02:09:56.681396 8365 ts_manager.cc:194] Registered new tserver with Master: 39398a776dc64ce1a8d3753a1e3ab8d6 (127.5.9.130:36273)
I20260809 02:09:56.681852 8365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:48967
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:09:56.702709 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:37441
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:56.789810 8708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:56.790057 8708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:56.790083 8708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:56.791323 8708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:56.791373 8708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:56.791400 8708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:09:56.792969 8708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:37441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:56.793256 8708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:56.793512 8708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:56.798707 8714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:56.798766 8708 server_base.cc:1034] running on GCE node
W20260809 02:09:56.798712 8717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:56.798703 8713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:56.799125 8708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:56.799407 8708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:56.800609 8708 hybrid_clock.cc:648] HybridClock initialized: now 1786241396800581 us; error 50 us; skew 500 ppm
Aug 09 02:09:56 dist-test-slave-p3t5 krb5kdc[8312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241396, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:56.803352 8708 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:09:56.803840 8708 webserver.cc:460] Webserver started at http://127.5.9.131:40677/ using document root <none> and password file <none>
I20260809 02:09:56.804085 8708 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:56.804157 8708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:56.804320 8708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:56.805485 8708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "02383922fb574e808d2d680d05df8e14"
format_stamp: "Formatted at 2026-08-09 02:09:56 on dist-test-slave-p3t5"
server_key: "97c20463484f01a5ec786e7fb5f91741"
server_key_iv: "9d8417db1746067936d2b4f9d29a7ae1"
server_key_version: "encryptionkey@0"
I20260809 02:09:56.805958 8708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "02383922fb574e808d2d680d05df8e14"
format_stamp: "Formatted at 2026-08-09 02:09:56 on dist-test-slave-p3t5"
server_key: "97c20463484f01a5ec786e7fb5f91741"
server_key_iv: "9d8417db1746067936d2b4f9d29a7ae1"
server_key_version: "encryptionkey@0"
I20260809 02:09:56.807423 8708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:56.808372 8724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:56.808607 8708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:56.808717 8708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "02383922fb574e808d2d680d05df8e14"
format_stamp: "Formatted at 2026-08-09 02:09:56 on dist-test-slave-p3t5"
server_key: "97c20463484f01a5ec786e7fb5f91741"
server_key_iv: "9d8417db1746067936d2b4f9d29a7ae1"
server_key_version: "encryptionkey@0"
I20260809 02:09:56.808856 8708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:56.827075 8708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:56.829706 8708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:56.829849 8708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:56.830150 8708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:09:56.830523 8708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:56.830582 8708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:56.830627 8708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:56.830655 8708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:56.837455 8708 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:45955
I20260809 02:09:56.837499 8842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:45955 every 8 connection(s)
I20260809 02:09:56.837920 8708 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:09:56.839579 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 8708
I20260809 02:09:56.839692 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:09:56.839954 5158 external_mini_cluster.cc:1428] Setting key bde82e4962652b8fc65244559fd33d6b
Aug 09 02:09:56 dist-test-slave-p3t5 krb5kdc[8312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241396, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:56.848718 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:56.838764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:56009 (local address 127.5.9.190:37441)
0809 02:09:56.838856 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:56.838859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:56.839068 (+ 209us) server_negotiation.cc:407] Connection header received
0809 02:09:56.839880 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:56.839882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:56.839897 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:56.839932 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:56.841396 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:56.841977 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:56.843041 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:56.843265 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:56.843351 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:56.845689 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:56.845690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:56.845691 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:56.845699 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:56.847159 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:56.847636 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:56.847637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:56.847638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:56.847709 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:56.847986 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:56.847987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:56.847988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:56.848119 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:56.848201 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:56.848501 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 02:09:56.848593 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:09:56.848923 8845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:56.838715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:37441 (local address 127.5.9.131:56009)
0809 02:09:56.838932 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:56.838972 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:56.839727 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:56.840156 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:56.840160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:56.841006 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:56.841286 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:56.841292 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:56.842077 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:56.842078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:56.842939 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:56.842942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:56.843361 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:56.843363 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:56.843500 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:56.844058 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:56.844067 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:56.845533 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:56.847276 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:56.847285 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:56.847287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:56.847550 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:56.847783 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:56.847784 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:56.847784 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:56.847911 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:56.848191 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:56.848198 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:56.848346 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 02:09:56.848467 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 02:09:56.848649 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":106,"threads_started":1}
I20260809 02:09:56.849479 8843 heartbeater.cc:347] Connected to a master server at 127.5.9.190:37441
I20260809 02:09:56.849582 8843 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:56.849804 8843 heartbeater.cc:511] Master 127.5.9.190:37441 requested a full tablet report, sending...
I20260809 02:09:56.850200 8365 ts_manager.cc:194] Registered new tserver with Master: 02383922fb574e808d2d680d05df8e14 (127.5.9.131:45955)
I20260809 02:09:56.850562 8365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:56009
I20260809 02:09:56.852444 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:09:56 dist-test-slave-p3t5 krb5kdc[8312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241396, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 02:09:56 dist-test-slave-p3t5 krb5kdc[8312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241396, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:56.879720 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:56.871224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32972 (local address 127.5.9.190:37441)
0809 02:09:56.871305 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:56.871308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:56.871453 (+ 145us) server_negotiation.cc:407] Connection header received
0809 02:09:56.871528 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:56.871529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:56.871539 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:56.871592 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:56.872297 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:56.872804 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:56.873648 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:56.873875 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:56.873932 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:56.876869 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:56.876870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:56.876871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:56.876880 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:56.878342 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:56.878715 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:56.878717 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:56.878717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:56.878783 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:56.878981 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:56.878981 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:56.878982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:56.879148 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:56.879232 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:56.879510 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 02:09:56.879598 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 02:09:56.880049 8365 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:32972
I20260809 02:09:56.880676 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 8431
I20260809 02:09:56.887537 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 8570
I20260809 02:09:56.893258 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 8708
I20260809 02:09:56.899027 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 8326
2026-08-09T02:09:56Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1950 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:09:56 dist-test-slave-p3t5 krb5kdc[8856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:09:56 dist-test-slave-p3t5 krb5kdc[8856](info): set up 2 sockets
Aug 09 02:09:56 dist-test-slave-p3t5 krb5kdc[8856](info): commencing operation
krb5kdc: starting...
W20260809 02:09:57.974833 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:09:58 dist-test-slave-p3t5 krb5kdc[8856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:09:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:09:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:09:58.126684 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:39095
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:39095
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:58.216836 8870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:58.217113 8870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:58.217141 8870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:58.218461 8870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:09:58.218530 8870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:58.218546 8870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:58.218564 8870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:09:58.218576 8870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:09:58.220319 8870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:39095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:39095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:58.220575 8870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:58.220791 8870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:58.226691 8875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:58.226655 8877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:58.226660 8879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:58.226758 8870 server_base.cc:1034] running on GCE node
I20260809 02:09:58.227365 8870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:58.227721 8870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:58.228911 8870 hybrid_clock.cc:648] HybridClock initialized: now 1786241398228882 us; error 41 us; skew 500 ppm
Aug 09 02:09:58 dist-test-slave-p3t5 krb5kdc[8856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241398, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:58.232421 8870 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:09:58.233002 8870 webserver.cc:460] Webserver started at http://127.5.9.190:44673/ using document root <none> and password file <none>
I20260809 02:09:58.233302 8870 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:58.233381 8870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:58.233551 8870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:58.234913 8870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "cf2211528424429583438424e7b48eb3"
format_stamp: "Formatted at 2026-08-09 02:09:58 on dist-test-slave-p3t5"
server_key: "3b15612c230fddfa2dc284989eecfcd7"
server_key_iv: "ed67b24dc009be09b6dede2d42997f84"
server_key_version: "encryptionkey@0"
I20260809 02:09:58.235351 8870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "cf2211528424429583438424e7b48eb3"
format_stamp: "Formatted at 2026-08-09 02:09:58 on dist-test-slave-p3t5"
server_key: "3b15612c230fddfa2dc284989eecfcd7"
server_key_iv: "ed67b24dc009be09b6dede2d42997f84"
server_key_version: "encryptionkey@0"
I20260809 02:09:58.237025 8870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:09:58.238111 8886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:58.238436 8870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:09:58.238566 8870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "cf2211528424429583438424e7b48eb3"
format_stamp: "Formatted at 2026-08-09 02:09:58 on dist-test-slave-p3t5"
server_key: "3b15612c230fddfa2dc284989eecfcd7"
server_key_iv: "ed67b24dc009be09b6dede2d42997f84"
server_key_version: "encryptionkey@0"
I20260809 02:09:58.238662 8870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:58.261775 8870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:58.265288 8870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:58.265527 8870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:58.271598 8870 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:39095
I20260809 02:09:58.271916 8941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:39095 every 8 connection(s)
I20260809 02:09:58.272090 8870 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:09:58.272931 8942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:09:58.274015 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 8870
I20260809 02:09:58.274132 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:09:58.274343 5158 external_mini_cluster.cc:1428] Setting key 113f4b060925f7d007e8aeb2b4c6d6fd
I20260809 02:09:58.276840 8942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3: Bootstrap starting.
I20260809 02:09:58.277827 8942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3: Neither blocks nor log segments found. Creating new log.
I20260809 02:09:58.278227 8942 log.cc:824] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3: Log is configured to *not* fsync() on all Append() calls
I20260809 02:09:58.279158 8942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3: No bootstrap required, opened a new log
Aug 09 02:09:58 dist-test-slave-p3t5 krb5kdc[8856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:58.282125 8942 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2211528424429583438424e7b48eb3" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39095 } }
I20260809 02:09:58.282354 8942 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:09:58.282389 8942 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf2211528424429583438424e7b48eb3, State: Initialized, Role: FOLLOWER
I20260809 02:09:58.282612 8942 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2211528424429583438424e7b48eb3" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39095 } }
I20260809 02:09:58.282689 8942 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:09:58.282737 8942 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:09:58.282801 8942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:09:58.283684 8942 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2211528424429583438424e7b48eb3" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39095 } }
I20260809 02:09:58.283828 8942 leader_election.cc:302] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf2211528424429583438424e7b48eb3; no voters:
I20260809 02:09:58.284065 8942 leader_election.cc:288] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:09:58.284252 8947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:09:58.284375 8942 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:09:58.284452 8947 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [term 1 LEADER]: Becoming Leader. State: Replica: cf2211528424429583438424e7b48eb3, State: Running, Role: LEADER
I20260809 02:09:58.284564 8947 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2211528424429583438424e7b48eb3" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39095 } }
I20260809 02:09:58.284978 8948 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf2211528424429583438424e7b48eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2211528424429583438424e7b48eb3" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39095 } } }
I20260809 02:09:58.285090 8948 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:09:58.284967 8947 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf2211528424429583438424e7b48eb3. Latest consensus state: current_term: 1 leader_uuid: "cf2211528424429583438424e7b48eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2211528424429583438424e7b48eb3" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39095 } } }
I20260809 02:09:58.285342 8947 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:09:58.285498 8956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:09:58.286131 8956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:09:58.287926 8945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:58.274984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37714 (local address 127.5.9.190:39095)
0809 02:09:58.275174 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:58.275183 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:58.275212 (+ 29us) server_negotiation.cc:407] Connection header received
0809 02:09:58.275709 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:58.275717 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:58.275771 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:58.275900 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:58.276830 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.278013 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.279059 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.279544 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.279634 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:58.282801 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:58.282807 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:58.282807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:58.282823 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:58.285456 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:58.286064 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:58.286068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:58.286069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:58.286200 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:58.286663 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:58.286664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:58.286665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:58.287030 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:58.287203 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:58.287502 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 02:09:58.287669 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260809 02:09:58.288389 8956 catalog_manager.cc:1347] Generated new cluster ID: f4639ebbd2574897bd7b2e747bfdd353
I20260809 02:09:58.288496 8956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:09:58.294600 8956 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:09:58.295233 8956 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:09:58.309973 8956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf2211528424429583438424e7b48eb3: Generated new TSK 0
I20260809 02:09:58.310240 8956 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:09:58.354326 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:39095
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:58.450740 8968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:58.450991 8968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:58.451020 8968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:58.452338 8968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:58.452389 8968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:58.452418 8968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:09:58.454175 8968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:39095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:58.454421 8968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:58.454628 8968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:58.460726 8974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:58.460786 8968 server_base.cc:1034] running on GCE node
W20260809 02:09:58.460803 8973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:58.460747 8977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:58.461282 8968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:58.461596 8968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:58.462889 8968 hybrid_clock.cc:648] HybridClock initialized: now 1786241398462851 us; error 45 us; skew 500 ppm
Aug 09 02:09:58 dist-test-slave-p3t5 krb5kdc[8856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241398, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:58.466183 8968 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:09:58.466840 8968 webserver.cc:460] Webserver started at http://127.5.9.129:35361/ using document root <none> and password file <none>
I20260809 02:09:58.467116 8968 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:58.467171 8968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:58.467319 8968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:58.468878 8968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "3a22f3d5f56f47a5a1e5e88d6db71f95"
format_stamp: "Formatted at 2026-08-09 02:09:58 on dist-test-slave-p3t5"
server_key: "a62d40b88eb5c10735741d15fbf128c7"
server_key_iv: "67f65843d5300a2328dd2da6e32a8f89"
server_key_version: "encryptionkey@0"
I20260809 02:09:58.469338 8968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "3a22f3d5f56f47a5a1e5e88d6db71f95"
format_stamp: "Formatted at 2026-08-09 02:09:58 on dist-test-slave-p3t5"
server_key: "a62d40b88eb5c10735741d15fbf128c7"
server_key_iv: "67f65843d5300a2328dd2da6e32a8f89"
server_key_version: "encryptionkey@0"
I20260809 02:09:58.471489 8968 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 02:09:58.472608 8985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:58.472949 8968 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:09:58.473096 8968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "3a22f3d5f56f47a5a1e5e88d6db71f95"
format_stamp: "Formatted at 2026-08-09 02:09:58 on dist-test-slave-p3t5"
server_key: "a62d40b88eb5c10735741d15fbf128c7"
server_key_iv: "67f65843d5300a2328dd2da6e32a8f89"
server_key_version: "encryptionkey@0"
I20260809 02:09:58.473197 8968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:58.487778 8968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:58.490603 8968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:58.490813 8968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:58.491112 8968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:09:58.491658 8968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:58.491736 8968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:58.491776 8968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:58.491811 8968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:58.498994 8968 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:36009
I20260809 02:09:58.499107 9105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:36009 every 8 connection(s)
I20260809 02:09:58.499506 8968 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:09:58.501803 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 8968
I20260809 02:09:58.501983 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:09:58.502225 5158 external_mini_cluster.cc:1428] Setting key 8c076a92a49feb2d1f5e373fd1db02ed
Aug 09 02:09:58 dist-test-slave-p3t5 krb5kdc[8856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241398, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:58.512094 8945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:58.500461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:44455 (local address 127.5.9.190:39095)
0809 02:09:58.500550 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:58.500552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:58.500750 (+ 198us) server_negotiation.cc:407] Connection header received
0809 02:09:58.501815 (+ 1065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:58.501817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:58.501832 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:58.501920 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:58.502832 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.503641 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.504967 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.505209 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.505273 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:58.508445 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:58.508448 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:58.508448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:58.508458 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:58.510142 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:58.510806 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:58.510808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:58.510808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:58.510876 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:58.511272 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:58.511273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:58.511274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:58.511423 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:58.511480 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:58.511919 (+ 439us) server_negotiation.cc:299] Negotiation successful
0809 02:09:58.512018 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:09:58.512401 9108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:58.500399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39095 (local address 127.5.9.129:44455)
0809 02:09:58.500606 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:58.500650 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:58.501683 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:58.502053 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:58.502060 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:58.502447 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:58.502731 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:58.502739 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.503799 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.503800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:58.504842 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:58.504847 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.505319 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.505320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:58.505497 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:58.506253 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:58.506265 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:58.508282 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:58.510266 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:58.510281 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:58.510284 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:58.510684 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:58.510997 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:58.510999 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:58.510999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:58.511186 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:58.511498 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:58.511508 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:58.511717 (+ 209us) client_negotiation.cc:769] Sending connection context
0809 02:09:58.511857 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 02:09:58.512074 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":107,"threads_started":1}
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
I20260809 02:09:58.513059 9106 heartbeater.cc:347] Connected to a master server at 127.5.9.190:39095
I20260809 02:09:58.513159 9106 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:58.513374 9106 heartbeater.cc:511] Master 127.5.9.190:39095 requested a full tablet report, sending...
I20260809 02:09:58.513965 8905 ts_manager.cc:194] Registered new tserver with Master: 3a22f3d5f56f47a5a1e5e88d6db71f95 (127.5.9.129:36009)
I20260809 02:09:58.514727 8905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:44455
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:09:58.538551 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:39095
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:58.635591 9111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:58.635851 9111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:58.635880 9111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:58.637106 9111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:58.637151 9111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:58.637180 9111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:09:58.638875 9111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:39095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:58.639132 9111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:58.639334 9111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:58.645452 9116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:58.645464 9120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:58.645504 9111 server_base.cc:1034] running on GCE node
W20260809 02:09:58.645462 9117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:58.646035 9111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:58.646374 9111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:58.647559 9111 hybrid_clock.cc:648] HybridClock initialized: now 1786241398647525 us; error 50 us; skew 500 ppm
Aug 09 02:09:58 dist-test-slave-p3t5 krb5kdc[8856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241398, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:58.650756 9111 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:09:58.651384 9111 webserver.cc:460] Webserver started at http://127.5.9.130:35291/ using document root <none> and password file <none>
I20260809 02:09:58.651647 9111 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:58.651700 9111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:58.651821 9111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:58.653026 9111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "4b7d2037f7384b18acd82ad94aa62b65"
format_stamp: "Formatted at 2026-08-09 02:09:58 on dist-test-slave-p3t5"
server_key: "0d3b2c1a2b628f4522326d7b935f366a"
server_key_iv: "827278fe320896348e20aea7a32390ef"
server_key_version: "encryptionkey@0"
I20260809 02:09:58.653465 9111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "4b7d2037f7384b18acd82ad94aa62b65"
format_stamp: "Formatted at 2026-08-09 02:09:58 on dist-test-slave-p3t5"
server_key: "0d3b2c1a2b628f4522326d7b935f366a"
server_key_iv: "827278fe320896348e20aea7a32390ef"
server_key_version: "encryptionkey@0"
I20260809 02:09:58.655092 9111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:09:58.656174 9127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:58.656713 9111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:09:58.656843 9111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "4b7d2037f7384b18acd82ad94aa62b65"
format_stamp: "Formatted at 2026-08-09 02:09:58 on dist-test-slave-p3t5"
server_key: "0d3b2c1a2b628f4522326d7b935f366a"
server_key_iv: "827278fe320896348e20aea7a32390ef"
server_key_version: "encryptionkey@0"
I20260809 02:09:58.656948 9111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:58.670395 9111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:58.672998 9111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:58.673208 9111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:58.673512 9111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:09:58.674042 9111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:58.674105 9111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:58.674153 9111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:58.674181 9111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:58.680998 9111 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:44627
I20260809 02:09:58.681095 9243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:44627 every 8 connection(s)
I20260809 02:09:58.681484 9111 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:09:58.686316 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 9111
I20260809 02:09:58.686662 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:09:58.686933 5158 external_mini_cluster.cc:1428] Setting key 271106300148a56f08184751b9751c40
Aug 09 02:09:58 dist-test-slave-p3t5 krb5kdc[8856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241398, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:58.693656 8945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:58.682593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:52569 (local address 127.5.9.190:39095)
0809 02:09:58.682646 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:58.682649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:58.682979 (+ 330us) server_negotiation.cc:407] Connection header received
0809 02:09:58.684092 (+ 1113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:58.684094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:58.684107 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:58.684156 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:58.685049 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.685670 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.686883 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.687140 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.687233 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:58.690025 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:58.690027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:58.690028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:58.690049 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:58.691874 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:58.692360 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:58.692362 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:58.692363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:58.692447 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:58.692774 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:58.692776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:58.692776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:58.692961 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:58.693046 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:58.693346 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 02:09:58.693458 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:09:58.693737 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:58.682538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39095 (local address 127.5.9.130:52569)
0809 02:09:58.682820 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:58.682877 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:58.683939 (+ 1062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:58.684299 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:58.684303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:58.684694 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:58.684923 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:58.684928 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.685779 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.685780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:58.686693 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:58.686696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.687251 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.687252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:58.687415 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:58.688047 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:58.688057 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:58.689897 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:58.691981 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:58.691990 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:58.691992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:58.692263 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:58.692523 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:58.692524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:58.692524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:58.692681 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:58.693040 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:58.693047 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:58.693196 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 02:09:58.693293 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 02:09:58.693457 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":101,"threads_started":1}
I20260809 02:09:58.694496 9244 heartbeater.cc:347] Connected to a master server at 127.5.9.190:39095
I20260809 02:09:58.694631 9244 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:58.694864 9244 heartbeater.cc:511] Master 127.5.9.190:39095 requested a full tablet report, sending...
I20260809 02:09:58.695359 8905 ts_manager.cc:194] Registered new tserver with Master: 4b7d2037f7384b18acd82ad94aa62b65 (127.5.9.130:44627)
I20260809 02:09:58.695766 8905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:52569
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:09:58.722504 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:39095
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:09:58.814043 9249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:09:58.814347 9249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:09:58.814391 9249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:09:58.815714 9249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:09:58.815800 9249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:09:58.815850 9249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:09:58.817541 9249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:39095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:09:58.817898 9249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:09:58.818167 9249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:09:58.823608 9255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:58.823596 9257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:09:58.823596 9254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:09:58.823957 9249 server_base.cc:1034] running on GCE node
I20260809 02:09:58.824200 9249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:09:58.824479 9249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:09:58.825714 9249 hybrid_clock.cc:648] HybridClock initialized: now 1786241398825702 us; error 39 us; skew 500 ppm
Aug 09 02:09:58 dist-test-slave-p3t5 krb5kdc[8856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241398, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:09:58.828792 9249 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:09:58.829880 9249 webserver.cc:460] Webserver started at http://127.5.9.131:43485/ using document root <none> and password file <none>
I20260809 02:09:58.830170 9249 fs_manager.cc:359] Metadata directory not provided
I20260809 02:09:58.830247 9249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:09:58.830406 9249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:09:58.831615 9249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "c796da49e0d5458a988c735e77f28070"
format_stamp: "Formatted at 2026-08-09 02:09:58 on dist-test-slave-p3t5"
server_key: "51eccbfa7413f8d2bc7e5af3f46bc173"
server_key_iv: "2a9857eeb0d305fdfa74fb851309b52f"
server_key_version: "encryptionkey@0"
I20260809 02:09:58.832070 9249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "c796da49e0d5458a988c735e77f28070"
format_stamp: "Formatted at 2026-08-09 02:09:58 on dist-test-slave-p3t5"
server_key: "51eccbfa7413f8d2bc7e5af3f46bc173"
server_key_iv: "2a9857eeb0d305fdfa74fb851309b52f"
server_key_version: "encryptionkey@0"
I20260809 02:09:58.833988 9249 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 02:09:58.835049 9264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:58.835300 9249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:09:58.835430 9249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "c796da49e0d5458a988c735e77f28070"
format_stamp: "Formatted at 2026-08-09 02:09:58 on dist-test-slave-p3t5"
server_key: "51eccbfa7413f8d2bc7e5af3f46bc173"
server_key_iv: "2a9857eeb0d305fdfa74fb851309b52f"
server_key_version: "encryptionkey@0"
I20260809 02:09:58.835526 9249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:09:58.874632 9249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:09:58.877457 9249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:09:58.877671 9249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:09:58.877959 9249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:09:58.878319 9249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:09:58.878378 9249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:58.878417 9249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:09:58.878466 9249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:09:58.884871 9249 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:42523
I20260809 02:09:58.884961 9381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:42523 every 8 connection(s)
I20260809 02:09:58.885354 9249 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:09:58.890285 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 9249
I20260809 02:09:58.890409 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:09:58.890611 5158 external_mini_cluster.cc:1428] Setting key 7bc6e1d05e39d2f8965470d9de41eb59
Aug 09 02:09:58 dist-test-slave-p3t5 krb5kdc[8856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241398, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:58.897430 8945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:58.886376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:55731 (local address 127.5.9.190:39095)
0809 02:09:58.886448 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:58.886451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:58.886815 (+ 364us) server_negotiation.cc:407] Connection header received
0809 02:09:58.887885 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:58.887887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:58.887901 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:58.887964 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:58.888676 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.889203 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.890535 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.890873 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.890965 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:58.893975 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:58.893976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:58.893976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:58.893985 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:58.895792 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:58.896273 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:58.896275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:58.896275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:58.896339 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:58.896699 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:58.896700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:58.896700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:58.896875 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:58.896935 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:58.897255 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 02:09:58.897352 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:09:58.897679 9384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:58.886378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39095 (local address 127.5.9.131:55731)
0809 02:09:58.886621 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 02:09:58.886664 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:09:58.887753 (+ 1089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:09:58.888052 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:09:58.888056 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:09:58.888354 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:09:58.888566 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:58.888570 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.889330 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.889331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:58.890394 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:09:58.890398 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.890995 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.890996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:09:58.891177 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:58.892074 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:09:58.892083 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:09:58.893837 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:09:58.895901 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:58.895911 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:58.895914 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:58.896189 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:58.896457 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:09:58.896471 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:09:58.896471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:09:58.896609 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:09:58.896945 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:09:58.896950 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:09:58.897097 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 02:09:58.897195 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 02:09:58.897354 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":82,"threads_started":1}
I20260809 02:09:58.898267 9382 heartbeater.cc:347] Connected to a master server at 127.5.9.190:39095
I20260809 02:09:58.898370 9382 heartbeater.cc:464] Registering TS with master...
I20260809 02:09:58.898557 9382 heartbeater.cc:511] Master 127.5.9.190:39095 requested a full tablet report, sending...
I20260809 02:09:58.898979 8905 ts_manager.cc:194] Registered new tserver with Master: c796da49e0d5458a988c735e77f28070 (127.5.9.131:42523)
I20260809 02:09:58.899360 8905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:55731
I20260809 02:09:58.903012 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:09:58 dist-test-slave-p3t5 krb5kdc[8856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:09:58 dist-test-slave-p3t5 krb5kdc[8856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.129@KRBTEST.COM
I20260809 02:09:58.932400 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:58.922079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42188 (local address 127.5.9.129:36009)
0809 02:09:58.922367 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:58.922372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:58.922387 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:09:58.922423 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:58.922425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:58.922440 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:58.922520 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:58.923516 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.924311 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.925175 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.925605 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.925796 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:58.928679 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:58.928694 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:58.928694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:58.928706 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:58.930962 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:58.931430 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:58.931433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:58.931433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:58.931502 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:58.931727 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:58.931728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:58.931729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:58.931905 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:58.932110 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:58.932158 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 02:09:58.932255 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":143,"threads_started":1}
Aug 09 02:09:58 dist-test-slave-p3t5 krb5kdc[8856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:09:58.944778 8945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:09:58.934235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37726 (local address 127.5.9.190:39095)
0809 02:09:58.934308 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 02:09:58.934310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:09:58.934380 (+ 70us) server_negotiation.cc:407] Connection header received
0809 02:09:58.934485 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:09:58.934486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:09:58.934497 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:09:58.934548 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:09:58.935056 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.935812 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.936930 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:09:58.937296 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:09:58.937394 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:09:58.940795 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:09:58.940797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:09:58.940798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:09:58.940809 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:09:58.943066 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:58.943581 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:58.943583 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:58.943583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:58.943679 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:09:58.944015 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:09:58.944016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:09:58.944017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:09:58.944208 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:09:58.944313 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 02:09:58.944551 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 02:09:58.944665 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 02:09:58.945015 8905 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37726
I20260809 02:09:58.945463 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 8968
I20260809 02:09:58.957598 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 9111
I20260809 02:09:58.966606 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 9249
I20260809 02:09:58.974071 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 8870
2026-08-09T02:09:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2075 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:09:59 dist-test-slave-p3t5 krb5kdc[9394](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:09:59 dist-test-slave-p3t5 krb5kdc[9394](info): set up 2 sockets
Aug 09 02:09:59 dist-test-slave-p3t5 krb5kdc[9394](info): commencing operation
krb5kdc: starting...
W20260809 02:10:00.078718 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:10:00 dist-test-slave-p3t5 krb5kdc[9394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:10:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:10:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:10:00.221474 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:44433
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:44433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:00.310246 9407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:00.310556 9407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:00.310603 9407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:00.311908 9407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:00.312000 9407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:00.312024 9407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:00.312047 9407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:00.312072 9407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:10:00.313822 9407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:44433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:44433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:00.314177 9407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:00.314453 9407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:00.319944 9413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:00.319957 9414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:00.320010 9416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:00.320184 9407 server_base.cc:1034] running on GCE node
I20260809 02:10:00.320616 9407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:00.321004 9407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:00.322198 9407 hybrid_clock.cc:648] HybridClock initialized: now 1786241400322178 us; error 51 us; skew 500 ppm
Aug 09 02:10:00 dist-test-slave-p3t5 krb5kdc[9394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:00.329522 9407 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:10:00.329983 9407 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:00.330066 9407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:00.330230 9407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:00.331467 9407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "b60546bbe5d8401d8bc2b76c5a59a6c1"
format_stamp: "Formatted at 2026-08-09 02:10:00 on dist-test-slave-p3t5"
server_key: "024070669d523b125a6c0da88f119b33"
server_key_iv: "904f8c66b94fbd347cf170e54134083d"
server_key_version: "encryptionkey@0"
I20260809 02:10:00.331871 9407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "b60546bbe5d8401d8bc2b76c5a59a6c1"
format_stamp: "Formatted at 2026-08-09 02:10:00 on dist-test-slave-p3t5"
server_key: "024070669d523b125a6c0da88f119b33"
server_key_iv: "904f8c66b94fbd347cf170e54134083d"
server_key_version: "encryptionkey@0"
I20260809 02:10:00.333313 9407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:10:00.334251 9422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:00.334470 9407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:10:00.334595 9407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "b60546bbe5d8401d8bc2b76c5a59a6c1"
format_stamp: "Formatted at 2026-08-09 02:10:00 on dist-test-slave-p3t5"
server_key: "024070669d523b125a6c0da88f119b33"
server_key_iv: "904f8c66b94fbd347cf170e54134083d"
server_key_version: "encryptionkey@0"
I20260809 02:10:00.334677 9407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:00.353572 9407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:00.356208 9407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:00.356339 9407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:00.361011 9407 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:44433
I20260809 02:10:00.361038 9476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:44433 every 8 connection(s)
I20260809 02:10:00.361369 9407 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:10:00.362524 9477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:00.365041 9477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1: Bootstrap starting.
I20260809 02:10:00.365741 9477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:00.366068 9477 log.cc:824] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:00.366948 9477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1: No bootstrap required, opened a new log
I20260809 02:10:00.368443 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 9407
I20260809 02:10:00.368566 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:10:00.368774 5158 external_mini_cluster.cc:1428] Setting key 286a5a4cb778113870462782a53bb119
I20260809 02:10:00.369062 9477 raft_consensus.cc:348] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b60546bbe5d8401d8bc2b76c5a59a6c1" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44433 } }
I20260809 02:10:00.369246 9477 raft_consensus.cc:374] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:00.369297 9477 raft_consensus.cc:729] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60546bbe5d8401d8bc2b76c5a59a6c1, State: Initialized, Role: FOLLOWER
I20260809 02:10:00.369472 9477 consensus_queue.cc:260] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b60546bbe5d8401d8bc2b76c5a59a6c1" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44433 } }
I20260809 02:10:00.369571 9477 raft_consensus.cc:388] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:10:00.369637 9477 raft_consensus.cc:482] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:10:00.369695 9477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:00.370741 9477 raft_consensus.cc:504] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b60546bbe5d8401d8bc2b76c5a59a6c1" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44433 } }
I20260809 02:10:00.370838 9477 leader_election.cc:302] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b60546bbe5d8401d8bc2b76c5a59a6c1; no voters:
I20260809 02:10:00.371070 9477 leader_election.cc:288] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:10:00.371110 9482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:00.371232 9482 raft_consensus.cc:686] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [term 1 LEADER]: Becoming Leader. State: Replica: b60546bbe5d8401d8bc2b76c5a59a6c1, State: Running, Role: LEADER
I20260809 02:10:00.371316 9482 consensus_queue.cc:237] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b60546bbe5d8401d8bc2b76c5a59a6c1" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44433 } }
I20260809 02:10:00.371362 9477 sys_catalog.cc:564] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:00.371680 9482 sys_catalog.cc:455] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b60546bbe5d8401d8bc2b76c5a59a6c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b60546bbe5d8401d8bc2b76c5a59a6c1" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44433 } } }
I20260809 02:10:00.371757 9482 sys_catalog.cc:458] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:00.371887 9482 sys_catalog.cc:455] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b60546bbe5d8401d8bc2b76c5a59a6c1. Latest consensus state: current_term: 1 leader_uuid: "b60546bbe5d8401d8bc2b76c5a59a6c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b60546bbe5d8401d8bc2b76c5a59a6c1" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44433 } } }
I20260809 02:10:00.371974 9482 sys_catalog.cc:458] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:00.372215 9486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:10:00.372946 9486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:10:00.374773 9486 catalog_manager.cc:1347] Generated new cluster ID: 8fa064d95cd243f9a0d73f95ccd592f7
I20260809 02:10:00.374837 9486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:10:00 dist-test-slave-p3t5 krb5kdc[9394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:10:00.383608 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:00.369422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43526 (local address 127.5.9.190:44433)
0809 02:10:00.369680 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:00.369691 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:00.369729 (+ 38us) server_negotiation.cc:407] Connection header received
0809 02:10:00.369857 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:00.369863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:00.369909 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:00.370056 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:00.370774 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.372397 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.373445 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.374089 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.374211 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:00.379859 (+ 5648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:00.379865 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:00.379866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:00.379889 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:00.381845 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:00.382258 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:00.382269 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:00.382270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:00.382382 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:00.382637 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:00.382638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:00.382638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:00.382917 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:00.383094 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:00.383199 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 02:10:00.383358 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":103,"threads_started":1}
I20260809 02:10:00.388449 9486 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:10:00.389205 9486 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:10:00.392970 9486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1: Generated new TSK 0
I20260809 02:10:00.393184 9486 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:10:00.433535 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:44433
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:00.522184 9504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:00.522432 9504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:00.522460 9504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:00.523751 9504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:00.523805 9504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:00.523834 9504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:10:00.525496 9504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:44433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:00.525813 9504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:00.526031 9504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:00.531522 9512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:00.531517 9510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:00.531522 9509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:00.531870 9504 server_base.cc:1034] running on GCE node
I20260809 02:10:00.532085 9504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:00.532351 9504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:00.533562 9504 hybrid_clock.cc:648] HybridClock initialized: now 1786241400533533 us; error 40 us; skew 500 ppm
Aug 09 02:10:00 dist-test-slave-p3t5 krb5kdc[9394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:00.536432 9504 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:10:00.536839 9504 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:00.536893 9504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:00.536994 9504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:00.538157 9504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "d47d8292e51947e69f508f3596c24274"
format_stamp: "Formatted at 2026-08-09 02:10:00 on dist-test-slave-p3t5"
server_key: "fcbe710c02adb60eb06dd98e4f49d356"
server_key_iv: "1be0f6271fac705afe96304e8a165c77"
server_key_version: "encryptionkey@0"
I20260809 02:10:00.538493 9504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "d47d8292e51947e69f508f3596c24274"
format_stamp: "Formatted at 2026-08-09 02:10:00 on dist-test-slave-p3t5"
server_key: "fcbe710c02adb60eb06dd98e4f49d356"
server_key_iv: "1be0f6271fac705afe96304e8a165c77"
server_key_version: "encryptionkey@0"
I20260809 02:10:00.539991 9504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:00.540894 9519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:00.541141 9504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:00.541273 9504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "d47d8292e51947e69f508f3596c24274"
format_stamp: "Formatted at 2026-08-09 02:10:00 on dist-test-slave-p3t5"
server_key: "fcbe710c02adb60eb06dd98e4f49d356"
server_key_iv: "1be0f6271fac705afe96304e8a165c77"
server_key_version: "encryptionkey@0"
I20260809 02:10:00.541368 9504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:00.561663 9504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:00.564841 9504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:00.565047 9504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:00.565383 9504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:00.565855 9504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:00.565910 9504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:00.565955 9504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:00.565969 9504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:00.573323 9504 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:34131
I20260809 02:10:00.573364 9637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:34131 every 8 connection(s)
I20260809 02:10:00.573758 9504 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:10:00 dist-test-slave-p3t5 krb5kdc[9394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:10:00.580677 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 9504
I20260809 02:10:00.580778 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:10:00.581001 5158 external_mini_cluster.cc:1428] Setting key d6945b2628879c249a47f3a46563f97c
I20260809 02:10:00.584493 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:00.574592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:51243 (local address 127.5.9.190:44433)
0809 02:10:00.574658 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:00.574660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:00.574945 (+ 285us) server_negotiation.cc:407] Connection header received
0809 02:10:00.575432 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:00.575433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:00.575447 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:00.575502 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:00.576499 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.577016 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.578155 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.578368 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.578429 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:00.580988 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:00.580989 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:00.580990 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:00.581001 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:00.582897 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:00.583338 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:00.583339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:00.583340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:00.583415 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:00.583664 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:00.583665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:00.583665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:00.583845 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:00.583926 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:00.584332 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 02:10:00.584420 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 02:10:00.584794 9640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:00.574581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:44433 (local address 127.5.9.129:51243)
0809 02:10:00.574810 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:00.574851 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:00.575346 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:00.575562 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:00.575565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:00.575859 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:00.576408 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:00.576414 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.577110 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.577111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:00.578071 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:00.578074 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.578451 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.578452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:00.578558 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:00.579176 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:00.579185 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:00.580880 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:00.582996 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:00.583005 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:00.583007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:00.583257 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:00.583485 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:00.583485 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:00.583486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:00.583599 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:00.583942 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:00.583949 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:00.584150 (+ 201us) client_negotiation.cc:769] Sending connection context
0809 02:10:00.584294 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 02:10:00.584502 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":114,"threads_started":1}
I20260809 02:10:00.585363 9638 heartbeater.cc:347] Connected to a master server at 127.5.9.190:44433
I20260809 02:10:00.585489 9638 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:00.585719 9638 heartbeater.cc:511] Master 127.5.9.190:44433 requested a full tablet report, sending...
I20260809 02:10:00.586319 9439 ts_manager.cc:194] Registered new tserver with Master: d47d8292e51947e69f508f3596c24274 (127.5.9.129:34131)
I20260809 02:10:00.586997 9439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:51243
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:10:00.614593 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:44433
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:00.700794 9644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:00.701012 9644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:00.701038 9644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:00.702271 9644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:00.702316 9644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:00.702345 9644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:10:00.703841 9644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:44433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:00.704048 9644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:00.704241 9644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:00.709514 9649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:00.709518 9652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:00.709514 9650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:00.709828 9644 server_base.cc:1034] running on GCE node
I20260809 02:10:00.710083 9644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:00.710345 9644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:00.711508 9644 hybrid_clock.cc:648] HybridClock initialized: now 1786241400711485 us; error 39 us; skew 500 ppm
Aug 09 02:10:00 dist-test-slave-p3t5 krb5kdc[9394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:00.714295 9644 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:10:00.714725 9644 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:00.714778 9644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:00.714967 9644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:00.716127 9644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "b3d287f3343a4eb491f021907d12f0d1"
format_stamp: "Formatted at 2026-08-09 02:10:00 on dist-test-slave-p3t5"
server_key: "b78689e32b3510007796afdb20affd8a"
server_key_iv: "c0dada45c4f895d8bc4f6908ac31d4ab"
server_key_version: "encryptionkey@0"
I20260809 02:10:00.716519 9644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "b3d287f3343a4eb491f021907d12f0d1"
format_stamp: "Formatted at 2026-08-09 02:10:00 on dist-test-slave-p3t5"
server_key: "b78689e32b3510007796afdb20affd8a"
server_key_iv: "c0dada45c4f895d8bc4f6908ac31d4ab"
server_key_version: "encryptionkey@0"
I20260809 02:10:00.717999 9644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:10:00.718868 9659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:00.719085 9644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:10:00.719183 9644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "b3d287f3343a4eb491f021907d12f0d1"
format_stamp: "Formatted at 2026-08-09 02:10:00 on dist-test-slave-p3t5"
server_key: "b78689e32b3510007796afdb20affd8a"
server_key_iv: "c0dada45c4f895d8bc4f6908ac31d4ab"
server_key_version: "encryptionkey@0"
I20260809 02:10:00.719246 9644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:00.744002 9644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:00.746794 9644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:00.746958 9644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:00.747184 9644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:00.747551 9644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:00.747593 9644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:00.747618 9644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:00.747632 9644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:00.753854 9644 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:33357
I20260809 02:10:00.753890 9778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:33357 every 8 connection(s)
I20260809 02:10:00.754195 9644 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:10:00 dist-test-slave-p3t5 krb5kdc[9394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:10:00.761134 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 9644
I20260809 02:10:00.761255 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:10:00.761482 5158 external_mini_cluster.cc:1428] Setting key 9daca3c9011f3a2a5dbc85f10a85d7a0
I20260809 02:10:00.764914 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:00.754994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:32967 (local address 127.5.9.190:44433)
0809 02:10:00.755063 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:00.755064 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:00.755295 (+ 231us) server_negotiation.cc:407] Connection header received
0809 02:10:00.755862 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:00.755864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:00.755876 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:00.755928 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:00.756967 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.757512 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.758601 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.758810 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.758895 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:00.761434 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:00.761437 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:00.761437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:00.761453 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:00.763124 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:00.763745 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:00.763746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:00.763747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:00.763814 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:00.764115 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:00.764115 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:00.764116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:00.764280 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:00.764375 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:00.764685 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 02:10:00.764804 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:10:00.765246 9781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:00.754953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:44433 (local address 127.5.9.130:32967)
0809 02:10:00.755163 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:00.755204 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:00.755744 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:00.756005 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:00.756009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:00.756309 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:00.756841 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:00.756847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.757640 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.757644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:00.758501 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:00.758503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.758910 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.758911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:00.759050 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:00.759623 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:00.759632 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:00.761292 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:00.763221 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:00.763230 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:00.763233 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:00.763632 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:00.763888 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:00.763888 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:00.763889 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:00.764051 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:00.764359 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:00.764367 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:00.764541 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 02:10:00.764635 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 02:10:00.764925 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":99,"threads_started":1}
I20260809 02:10:00.765837 9779 heartbeater.cc:347] Connected to a master server at 127.5.9.190:44433
I20260809 02:10:00.766031 9779 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:00.766223 9779 heartbeater.cc:511] Master 127.5.9.190:44433 requested a full tablet report, sending...
I20260809 02:10:00.766708 9439 ts_manager.cc:194] Registered new tserver with Master: b3d287f3343a4eb491f021907d12f0d1 (127.5.9.130:33357)
I20260809 02:10:00.767169 9439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:32967
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:10:00.793123 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:44433
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:00.883179 9784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:00.883409 9784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:00.883435 9784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:00.884670 9784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:00.884722 9784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:00.884750 9784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:10:00.886948 9784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:44433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:00.887247 9784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:00.887511 9784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:00.893035 9789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:00.893157 9784 server_base.cc:1034] running on GCE node
W20260809 02:10:00.893035 9792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:00.893182 9790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:00.893484 9784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:00.893784 9784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:00.894974 9784 hybrid_clock.cc:648] HybridClock initialized: now 1786241400894987 us; error 56 us; skew 500 ppm
Aug 09 02:10:00 dist-test-slave-p3t5 krb5kdc[9394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:00.897713 9784 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:10:00.898147 9784 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:00.898224 9784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:00.898399 9784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:00.899556 9784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "f6532b26486447ef9543893256471174"
format_stamp: "Formatted at 2026-08-09 02:10:00 on dist-test-slave-p3t5"
server_key: "47bd3634c88d5db9fa306b890fdeb67c"
server_key_iv: "fee8f0f8074b01e75820f02e415da0e7"
server_key_version: "encryptionkey@0"
I20260809 02:10:00.899952 9784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "f6532b26486447ef9543893256471174"
format_stamp: "Formatted at 2026-08-09 02:10:00 on dist-test-slave-p3t5"
server_key: "47bd3634c88d5db9fa306b890fdeb67c"
server_key_iv: "fee8f0f8074b01e75820f02e415da0e7"
server_key_version: "encryptionkey@0"
I20260809 02:10:00.901376 9784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:10:00.902226 9799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:00.902412 9784 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:10:00.902524 9784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "f6532b26486447ef9543893256471174"
format_stamp: "Formatted at 2026-08-09 02:10:00 on dist-test-slave-p3t5"
server_key: "47bd3634c88d5db9fa306b890fdeb67c"
server_key_iv: "fee8f0f8074b01e75820f02e415da0e7"
server_key_version: "encryptionkey@0"
I20260809 02:10:00.902607 9784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:00.913419 9784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:00.915563 9784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:00.915740 9784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:00.916002 9784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:00.916378 9784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:00.916433 9784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:00.916466 9784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:00.916487 9784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:00.923081 9784 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:37005
I20260809 02:10:00.923100 9915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:37005 every 8 connection(s)
I20260809 02:10:00.923501 9784 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:10:00.929545 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 9784
I20260809 02:10:00.929687 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:10:00.929916 5158 external_mini_cluster.cc:1428] Setting key 6d971c1ee2a77793d01a41a325f49c56
Aug 09 02:10:00 dist-test-slave-p3t5 krb5kdc[9394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:10:00.934924 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:00.924380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:58847 (local address 127.5.9.190:44433)
0809 02:10:00.924444 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:00.924446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:00.924717 (+ 271us) server_negotiation.cc:407] Connection header received
0809 02:10:00.925204 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:00.925205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:00.925217 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:00.925287 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:00.926449 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.927015 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.928272 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.928496 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.928559 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:00.931495 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:00.931497 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:00.931497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:00.931505 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:00.933119 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:00.933553 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:00.933554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:00.933554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:00.933908 (+ 354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:00.934164 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:00.934165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:00.934165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:00.934301 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:00.934368 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:00.934736 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 02:10:00.934822 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:10:00.935087 9918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:00.924331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:44433 (local address 127.5.9.131:58847)
0809 02:10:00.924535 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:00.924583 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:00.925099 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:00.925386 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:00.925391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:00.925774 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:00.926329 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:00.926336 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.927134 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.927135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:00.928178 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:00.928180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.928586 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.928588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:00.928704 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:00.929369 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:00.929381 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:00.931383 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:00.933209 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:00.933216 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:00.933219 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:00.933466 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:00.933976 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:00.933977 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:00.933977 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:00.934097 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:00.934405 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:00.934412 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:00.934572 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 02:10:00.934678 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 02:10:00.934831 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":121,"threads_started":1}
I20260809 02:10:00.935792 9916 heartbeater.cc:347] Connected to a master server at 127.5.9.190:44433
I20260809 02:10:00.935901 9916 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:00.936038 9916 heartbeater.cc:511] Master 127.5.9.190:44433 requested a full tablet report, sending...
I20260809 02:10:00.936502 9439 ts_manager.cc:194] Registered new tserver with Master: f6532b26486447ef9543893256471174 (127.5.9.131:37005)
I20260809 02:10:00.936923 9439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:58847
I20260809 02:10:00.942333 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:10:00.950953 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:00.943918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43530 (local address 127.5.9.190:44433)
0809 02:10:00.943994 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:00.943998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:00.944130 (+ 132us) server_negotiation.cc:407] Connection header received
0809 02:10:00.944254 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:00.944255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:00.944268 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:00.944355 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:00.945065 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.945719 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.946561 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.946790 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.946912 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:00.948336 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:00.948338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:00.948338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:00.948347 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:00.949661 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:00.950042 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:00.950044 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:00.950044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:00.950113 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:00.950330 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:00.950331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:00.950332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:00.950504 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:00.950619 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:00.950745 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 02:10:00.950840 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:10:00.952718 9439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:10:00.954567 9439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:10:00.962139 9927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:00.957308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:37005 (local address 127.0.0.1:53006)
0809 02:10:00.957899 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:00.957909 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:00.958026 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:00.958462 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:00.958463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:00.958465 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:00.958527 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:00.958530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.959821 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.959822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:00.960771 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:00.960775 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.961747 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.961748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:00.961849 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:00.961865 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 02:10:00.961976 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 02:10:00.962041 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":77,"threads_started":1}
I20260809 02:10:00.962212 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:00.957670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52796 (local address 127.5.9.129:34131)
0809 02:10:00.957858 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:00.957862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:00.958222 (+ 360us) server_negotiation.cc:407] Connection header received
0809 02:10:00.958316 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:00.958319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:00.958336 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:00.958414 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:00.958635 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.959341 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.961047 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.961831 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.961926 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:00.961987 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:00.962080 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 02:10:00.962121 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":84,"threads_started":1}
I20260809 02:10:00.962522 9932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:00.957751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53006 (local address 127.5.9.131:37005)
0809 02:10:00.957945 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:00.957949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:00.957960 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:10:00.958343 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:00.958345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:00.958363 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:00.958440 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:00.958619 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.959711 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.960865 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.961652 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.961727 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:00.961804 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:00.962383 (+ 579us) server_negotiation.cc:299] Negotiation successful
0809 02:10:00.962431 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":95,"threads_started":1}
I20260809 02:10:00.962862 9929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:00.957316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:34131 (local address 127.0.0.1:52796)
0809 02:10:00.958157 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:00.958168 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:00.958221 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:00.958509 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:00.958510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:00.958512 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:00.958543 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:00.958544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.959467 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.959468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:00.960952 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:00.960953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.961929 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.961930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:00.961997 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:00.962002 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:10:00.962063 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 02:10:00.962112 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"mutex_wait_us":34,"thread_start_us":108,"threads_started":1}
I20260809 02:10:00.962965 9928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:00.957308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:33357 (local address 127.0.0.1:48850)
0809 02:10:00.957657 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:00.957704 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:00.957782 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:00.958077 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:00.958081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:00.958085 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:00.958185 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:00.958187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.959603 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.959604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:00.960948 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:00.960952 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.962718 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.962720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:00.962790 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:00.962796 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:10:00.962849 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 02:10:00.962892 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":77,"spinlock_wait_cycles":9088,"thread_start_us":133,"threads_started":1}
I20260809 02:10:00.963279 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:00.957515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48850 (local address 127.5.9.130:33357)
0809 02:10:00.957753 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:00.957759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:00.957773 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:10:00.957902 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:00.957904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:00.957922 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:00.958041 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:00.958295 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.959497 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.961736 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:00.962544 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:00.962616 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:00.962681 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:00.963146 (+ 465us) server_negotiation.cc:299] Negotiation successful
0809 02:10:00.963194 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":131,"threads_started":1}
I20260809 02:10:00.963292 9849 tablet_service.cc:1467] Processing CreateTablet for tablet b07f23026d9545be9c8ac0c8cb30405c (DEFAULT_TABLE table=test-table [id=843f3853c6164377b6e6e7c9014ab013]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:00.963594 9568 tablet_service.cc:1467] Processing CreateTablet for tablet b07f23026d9545be9c8ac0c8cb30405c (DEFAULT_TABLE table=test-table [id=843f3853c6164377b6e6e7c9014ab013]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:00.963726 9849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b07f23026d9545be9c8ac0c8cb30405c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:00.963979 9568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b07f23026d9545be9c8ac0c8cb30405c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:00.963904 9709 tablet_service.cc:1467] Processing CreateTablet for tablet b07f23026d9545be9c8ac0c8cb30405c (DEFAULT_TABLE table=test-table [id=843f3853c6164377b6e6e7c9014ab013]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:00.964190 9709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b07f23026d9545be9c8ac0c8cb30405c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:00.966941 9933 tablet_bootstrap.cc:492] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274: Bootstrap starting.
I20260809 02:10:00.967612 9934 tablet_bootstrap.cc:492] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174: Bootstrap starting.
I20260809 02:10:00.967618 9935 tablet_bootstrap.cc:492] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1: Bootstrap starting.
I20260809 02:10:00.967905 9933 tablet_bootstrap.cc:654] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:00.968269 9934 tablet_bootstrap.cc:654] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:00.968354 9935 tablet_bootstrap.cc:654] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:00.968688 9933 log.cc:824] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:00.968878 9934 log.cc:824] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:00.969063 9935 log.cc:824] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:00.969718 9933 tablet_bootstrap.cc:492] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274: No bootstrap required, opened a new log
I20260809 02:10:00.969802 9933 ts_tablet_manager.cc:1397] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:10:00.969908 9934 tablet_bootstrap.cc:492] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174: No bootstrap required, opened a new log
I20260809 02:10:00.969944 9935 tablet_bootstrap.cc:492] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1: No bootstrap required, opened a new log
I20260809 02:10:00.969985 9934 ts_tablet_manager.cc:1397] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:10:00.970036 9935 ts_tablet_manager.cc:1397] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:10:00.971660 9935 raft_consensus.cc:348] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6532b26486447ef9543893256471174" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37005 } } peers { permanent_uuid: "b3d287f3343a4eb491f021907d12f0d1" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33357 } } peers { permanent_uuid: "d47d8292e51947e69f508f3596c24274" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 34131 } }
I20260809 02:10:00.971836 9935 raft_consensus.cc:374] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:00.971881 9935 raft_consensus.cc:729] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3d287f3343a4eb491f021907d12f0d1, State: Initialized, Role: FOLLOWER
I20260809 02:10:00.972004 9935 consensus_queue.cc:260] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6532b26486447ef9543893256471174" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37005 } } peers { permanent_uuid: "b3d287f3343a4eb491f021907d12f0d1" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33357 } } peers { permanent_uuid: "d47d8292e51947e69f508f3596c24274" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 34131 } }
I20260809 02:10:00.972018 9934 raft_consensus.cc:348] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6532b26486447ef9543893256471174" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37005 } } peers { permanent_uuid: "b3d287f3343a4eb491f021907d12f0d1" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33357 } } peers { permanent_uuid: "d47d8292e51947e69f508f3596c24274" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 34131 } }
I20260809 02:10:00.972018 9933 raft_consensus.cc:348] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6532b26486447ef9543893256471174" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37005 } } peers { permanent_uuid: "b3d287f3343a4eb491f021907d12f0d1" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33357 } } peers { permanent_uuid: "d47d8292e51947e69f508f3596c24274" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 34131 } }
I20260809 02:10:00.972190 9933 raft_consensus.cc:374] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:00.972190 9934 raft_consensus.cc:374] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:00.972234 9934 raft_consensus.cc:729] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6532b26486447ef9543893256471174, State: Initialized, Role: FOLLOWER
I20260809 02:10:00.972234 9933 raft_consensus.cc:729] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d47d8292e51947e69f508f3596c24274, State: Initialized, Role: FOLLOWER
I20260809 02:10:00.972350 9933 consensus_queue.cc:260] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6532b26486447ef9543893256471174" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37005 } } peers { permanent_uuid: "b3d287f3343a4eb491f021907d12f0d1" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33357 } } peers { permanent_uuid: "d47d8292e51947e69f508f3596c24274" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 34131 } }
I20260809 02:10:00.972350 9934 consensus_queue.cc:260] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6532b26486447ef9543893256471174" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37005 } } peers { permanent_uuid: "b3d287f3343a4eb491f021907d12f0d1" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33357 } } peers { permanent_uuid: "d47d8292e51947e69f508f3596c24274" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 34131 } }
I20260809 02:10:00.972362 9935 ts_tablet_manager.cc:1428] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 02:10:00.972607 9779 heartbeater.cc:503] Master 127.5.9.190:44433 was elected leader, sending a full tablet report...
I20260809 02:10:00.972648 9933 ts_tablet_manager.cc:1428] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:10:00.972648 9934 ts_tablet_manager.cc:1428] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:10:00.972822 9916 heartbeater.cc:503] Master 127.5.9.190:44433 was elected leader, sending a full tablet report...
I20260809 02:10:00.972915 9638 heartbeater.cc:503] Master 127.5.9.190:44433 was elected leader, sending a full tablet report...
W20260809 02:10:01.004842 9780 tablet.cc:2369] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:10:01.074504 9639 tablet.cc:2369] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:10:01.149204 9942 raft_consensus.cc:482] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:10:01.149349 9942 raft_consensus.cc:504] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6532b26486447ef9543893256471174" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37005 } } peers { permanent_uuid: "b3d287f3343a4eb491f021907d12f0d1" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33357 } } peers { permanent_uuid: "d47d8292e51947e69f508f3596c24274" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 34131 } }
I20260809 02:10:01.149804 9942 leader_election.cc:288] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6532b26486447ef9543893256471174 (127.5.9.131:37005), b3d287f3343a4eb491f021907d12f0d1 (127.5.9.130:33357)
I20260809 02:10:01.153021 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.150119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:59457 (local address 127.5.9.130:33357)
0809 02:10:01.150172 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:01.150174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:01.150309 (+ 135us) server_negotiation.cc:407] Connection header received
0809 02:10:01.150383 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:01.150389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:01.150401 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:01.150475 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:01.150697 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:01.151152 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:01.152234 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:01.152613 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:01.152670 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:01.152679 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:01.152865 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 02:10:01.152915 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 02:10:01.153029 9932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.150170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:52371 (local address 127.5.9.131:37005)
0809 02:10:01.150229 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:01.150231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:01.150252 (+ 21us) server_negotiation.cc:407] Connection header received
0809 02:10:01.150375 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:01.150376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:01.150390 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:01.150474 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:01.150655 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:01.151154 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:01.152237 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:01.152624 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:01.152717 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:01.152726 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:01.152915 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 02:10:01.152969 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 02:10:01.153038 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.150043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:33357 (local address 127.5.9.129:59457)
0809 02:10:01.150213 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:01.150224 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:01.150303 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:01.150488 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:01.150489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:01.150491 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:01.150595 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:01.150596 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:01.151248 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:01.151249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:01.152130 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:01.152132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:01.152705 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:01.152706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:01.152788 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:01.152792 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:10:01.152835 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 02:10:01.152892 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":14,"thread_start_us":86,"threads_started":1}
I20260809 02:10:01.153040 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.150042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:37005 (local address 127.5.9.129:52371)
0809 02:10:01.150188 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:01.150201 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:01.150286 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:01.150483 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:01.150484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:01.150487 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:01.150575 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:01.150577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:01.151251 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:01.151252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:01.152128 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:01.152130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:01.152745 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:01.152746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:01.152809 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:01.152813 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:10:01.152887 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 02:10:01.152927 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":46,"thread_start_us":68,"threads_started":1}
I20260809 02:10:01.153414 9730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b07f23026d9545be9c8ac0c8cb30405c" candidate_uuid: "d47d8292e51947e69f508f3596c24274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3d287f3343a4eb491f021907d12f0d1" is_pre_election: true
I20260809 02:10:01.153637 9730 raft_consensus.cc:2454] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d47d8292e51947e69f508f3596c24274 in term 0.
I20260809 02:10:01.153637 9868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b07f23026d9545be9c8ac0c8cb30405c" candidate_uuid: "d47d8292e51947e69f508f3596c24274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6532b26486447ef9543893256471174" is_pre_election: true
I20260809 02:10:01.153771 9868 raft_consensus.cc:2454] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d47d8292e51947e69f508f3596c24274 in term 0.
I20260809 02:10:01.153909 9522 leader_election.cc:302] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3d287f3343a4eb491f021907d12f0d1, d47d8292e51947e69f508f3596c24274; no voters:
I20260809 02:10:01.154085 9942 raft_consensus.cc:2781] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:10:01.154147 9942 raft_consensus.cc:482] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:10:01.154193 9942 raft_consensus.cc:3037] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:01.154922 9942 raft_consensus.cc:504] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6532b26486447ef9543893256471174" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37005 } } peers { permanent_uuid: "b3d287f3343a4eb491f021907d12f0d1" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33357 } } peers { permanent_uuid: "d47d8292e51947e69f508f3596c24274" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 34131 } }
I20260809 02:10:01.155119 9942 leader_election.cc:288] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [CANDIDATE]: Term 1 election: Requested vote from peers f6532b26486447ef9543893256471174 (127.5.9.131:37005), b3d287f3343a4eb491f021907d12f0d1 (127.5.9.130:33357)
I20260809 02:10:01.155261 9868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b07f23026d9545be9c8ac0c8cb30405c" candidate_uuid: "d47d8292e51947e69f508f3596c24274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6532b26486447ef9543893256471174"
I20260809 02:10:01.155295 9730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b07f23026d9545be9c8ac0c8cb30405c" candidate_uuid: "d47d8292e51947e69f508f3596c24274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3d287f3343a4eb491f021907d12f0d1"
I20260809 02:10:01.155361 9868 raft_consensus.cc:3037] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:01.155383 9730 raft_consensus.cc:3037] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:01.156162 9730 raft_consensus.cc:2454] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d47d8292e51947e69f508f3596c24274 in term 1.
I20260809 02:10:01.156189 9868 raft_consensus.cc:2454] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d47d8292e51947e69f508f3596c24274 in term 1.
I20260809 02:10:01.156373 9522 leader_election.cc:302] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3d287f3343a4eb491f021907d12f0d1, d47d8292e51947e69f508f3596c24274; no voters:
I20260809 02:10:01.156486 9942 raft_consensus.cc:2781] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:01.156673 9942 raft_consensus.cc:686] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [term 1 LEADER]: Becoming Leader. State: Replica: d47d8292e51947e69f508f3596c24274, State: Running, Role: LEADER
I20260809 02:10:01.156798 9942 consensus_queue.cc:237] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6532b26486447ef9543893256471174" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37005 } } peers { permanent_uuid: "b3d287f3343a4eb491f021907d12f0d1" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33357 } } peers { permanent_uuid: "d47d8292e51947e69f508f3596c24274" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 34131 } }
I20260809 02:10:01.157831 9439 catalog_manager.cc:5495] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 reported cstate change: term changed from 0 to 1, leader changed from <none> to d47d8292e51947e69f508f3596c24274 (127.5.9.129). New cstate: current_term: 1 leader_uuid: "d47d8292e51947e69f508f3596c24274" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6532b26486447ef9543893256471174" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3d287f3343a4eb491f021907d12f0d1" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d47d8292e51947e69f508f3596c24274" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 34131 } health_report { overall_health: HEALTHY } } }
W20260809 02:10:01.174091 9917 tablet.cc:2369] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:10:01.190689 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.188216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52804 (local address 127.5.9.129:34131)
0809 02:10:01.188286 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:01.188293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:01.188302 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:10:01.188341 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:01.188342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:01.188362 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:01.188425 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:10:01.188572 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:01.189050 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:01.189912 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:01.190111 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:01.190170 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:01.190295 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:10:01.190404 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:10:01.190466 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:01.190545 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 02:10:01.190564 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:10:01.192857 9730 raft_consensus.cc:1261] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1 [term 1 FOLLOWER]: Refusing update from remote peer d47d8292e51947e69f508f3596c24274: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:01.192857 9868 raft_consensus.cc:1261] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174 [term 1 FOLLOWER]: Refusing update from remote peer d47d8292e51947e69f508f3596c24274: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:01.193296 9942 consensus_queue.cc:1035] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6532b26486447ef9543893256471174" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:01.193524 9945 consensus_queue.cc:1035] T b07f23026d9545be9c8ac0c8cb30405c P d47d8292e51947e69f508f3596c24274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3d287f3343a4eb491f021907d12f0d1" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:01.197204 9950 mvcc.cc:205] Tried to move back new op lower bound from 7316444779282358272 to 7316444779139235840. Current Snapshot: MvccSnapshot[applied={T|T < 7316444779282358272 or (T in {7316444779282358272})}]
I20260809 02:10:01.197341 9952 mvcc.cc:205] Tried to move back new op lower bound from 7316444779282358272 to 7316444779139235840. Current Snapshot: MvccSnapshot[applied={T|T < 7316444779282358272 or (T in {7316444779282358272})}]
I20260809 02:10:01.199626 9439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:01.199718 9439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:01.200601 9439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b60546bbe5d8401d8bc2b76c5a59a6c1: Sending DeleteTablet for 3 replicas of tablet b07f23026d9545be9c8ac0c8cb30405c
I20260809 02:10:01.200958 9709 tablet_service.cc:1514] Processing DeleteTablet for tablet b07f23026d9545be9c8ac0c8cb30405c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:01 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:48850
I20260809 02:10:01.200948 9568 tablet_service.cc:1514] Processing DeleteTablet for tablet b07f23026d9545be9c8ac0c8cb30405c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:01 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:52796
I20260809 02:10:01.201202 9849 tablet_service.cc:1514] Processing DeleteTablet for tablet b07f23026d9545be9c8ac0c8cb30405c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:01 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:53006
I20260809 02:10:01.201359 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 9504
I20260809 02:10:01.203446 9960 tablet_replica.cc:330] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1: stopping tablet replica
I20260809 02:10:01.203543 9960 raft_consensus.cc:2229] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:01.203625 9960 raft_consensus.cc:2258] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:01.204149 9960 ts_tablet_manager.cc:1905] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:01.205931 9960 ts_tablet_manager.cc:1918] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:01.206034 9960 log.cc:1196] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/b07f23026d9545be9c8ac0c8cb30405c
I20260809 02:10:01.206408 9960 ts_tablet_manager.cc:1939] T b07f23026d9545be9c8ac0c8cb30405c P b3d287f3343a4eb491f021907d12f0d1: Deleting consensus metadata
I20260809 02:10:01.207027 9425 catalog_manager.cc:4841] TS b3d287f3343a4eb491f021907d12f0d1 (127.5.9.130:33357): tablet b07f23026d9545be9c8ac0c8cb30405c (table test-table [id=843f3853c6164377b6e6e7c9014ab013]) successfully deleted
W20260809 02:10:01.208923 9423 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:34131 (error 108)
I20260809 02:10:01.209178 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 9644
I20260809 02:10:01.209236 9961 tablet_replica.cc:330] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174: stopping tablet replica
I20260809 02:10:01.209367 9961 raft_consensus.cc:2229] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:01.209482 9961 raft_consensus.cc:2258] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:01.210179 9927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.209369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:34131 (local address 127.0.0.1:52806)
0809 02:10:01.210077 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:01.210134 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:34131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":679}
W20260809 02:10:01.210333 9423 catalog_manager.cc:4584] TS d47d8292e51947e69f508f3596c24274 (127.5.9.129:34131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b07f23026d9545be9c8ac0c8cb30405c: Network error: Client connection negotiation failed: client connection to 127.5.9.129:34131: connect: Connection refused (error 111)
I20260809 02:10:01.211171 9961 ts_tablet_manager.cc:1905] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:01.212646 9961 ts_tablet_manager.cc:1918] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:01.212707 9961 log.cc:1196] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/b07f23026d9545be9c8ac0c8cb30405c
I20260809 02:10:01.212919 9961 ts_tablet_manager.cc:1939] T b07f23026d9545be9c8ac0c8cb30405c P f6532b26486447ef9543893256471174: Deleting consensus metadata
I20260809 02:10:01.213414 9426 catalog_manager.cc:4841] TS f6532b26486447ef9543893256471174 (127.5.9.131:37005): tablet b07f23026d9545be9c8ac0c8cb30405c (table test-table [id=843f3853c6164377b6e6e7c9014ab013]) successfully deleted
I20260809 02:10:01.216514 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 9784
I20260809 02:10:01.224627 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 9407
2026-08-09T02:10:01Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2250 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T02:10:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:10:01Z Disabled control of system clock
I20260809 02:10:01.257532 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:42277
--webserver_interface=127.5.9.190
--webserver_port=0
--builtin_ntp_servers=127.5.9.148:37977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:42277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 02:10:01.344247 9966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:01.344486 9966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:01.344511 9966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:01.345695 9966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:01.345738 9966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:01.345750 9966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:01.345763 9966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:01.345775 9966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:10:01.347147 9966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:42277
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:42277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:01.347380 9966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:01.347649 9966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:01.353040 9972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:01.353044 9971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:01.353049 9974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:01.353422 9966 server_base.cc:1034] running on GCE node
I20260809 02:10:01.353775 9966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:01.354131 9966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:01.355340 9966 hybrid_clock.cc:648] HybridClock initialized: now 1786241401355283 us; error 72 us; skew 500 ppm
I20260809 02:10:01.359795 9966 webserver.cc:460] Webserver started at http://127.5.9.190:35637/ using document root <none> and password file <none>
I20260809 02:10:01.360172 9966 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:01.360275 9966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:01.360461 9966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:01.361868 9966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "755a97f139ec4fa2b72d000d8430386a"
format_stamp: "Formatted at 2026-08-09 02:10:01 on dist-test-slave-p3t5"
server_key: "a814b61afd8bf3244a8e5268acaf4d93"
server_key_iv: "c68312a0d56b298b02e253830d61019a"
server_key_version: "encryptionkey@0"
I20260809 02:10:01.362313 9966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "755a97f139ec4fa2b72d000d8430386a"
format_stamp: "Formatted at 2026-08-09 02:10:01 on dist-test-slave-p3t5"
server_key: "a814b61afd8bf3244a8e5268acaf4d93"
server_key_iv: "c68312a0d56b298b02e253830d61019a"
server_key_version: "encryptionkey@0"
I20260809 02:10:01.363986 9966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:10:01.364926 9980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:01.365238 9966 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:01.365371 9966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "755a97f139ec4fa2b72d000d8430386a"
format_stamp: "Formatted at 2026-08-09 02:10:01 on dist-test-slave-p3t5"
server_key: "a814b61afd8bf3244a8e5268acaf4d93"
server_key_iv: "c68312a0d56b298b02e253830d61019a"
server_key_version: "encryptionkey@0"
I20260809 02:10:01.365466 9966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:01.369158 9966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:01.369477 9966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:01.369683 9966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:01.373997 9966 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:42277
I20260809 02:10:01.374069 10033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:42277 every 8 connection(s)
I20260809 02:10:01.374497 9966 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:10:01.375351 10034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:01.380056 10034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a: Bootstrap starting.
I20260809 02:10:01.380936 10034 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:01.381374 10034 log.cc:824] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:01.382387 10034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a: No bootstrap required, opened a new log
I20260809 02:10:01.383111 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 9966
I20260809 02:10:01.383232 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:10:01.383469 5158 external_mini_cluster.cc:1428] Setting key 823e9c30d7a1d90e60a47842868567b9
I20260809 02:10:01.385783 10034 raft_consensus.cc:348] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755a97f139ec4fa2b72d000d8430386a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42277 } }
I20260809 02:10:01.385999 10034 raft_consensus.cc:374] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:01.386041 10034 raft_consensus.cc:729] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755a97f139ec4fa2b72d000d8430386a, State: Initialized, Role: FOLLOWER
I20260809 02:10:01.386207 10034 consensus_queue.cc:260] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755a97f139ec4fa2b72d000d8430386a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42277 } }
I20260809 02:10:01.386323 10034 raft_consensus.cc:388] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:10:01.386379 10034 raft_consensus.cc:482] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:10:01.386417 10034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:01.387418 10034 raft_consensus.cc:504] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755a97f139ec4fa2b72d000d8430386a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42277 } }
I20260809 02:10:01.387518 10034 leader_election.cc:302] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 755a97f139ec4fa2b72d000d8430386a; no voters:
I20260809 02:10:01.387691 10034 leader_election.cc:288] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:10:01.387734 10039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:01.387873 10039 raft_consensus.cc:686] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [term 1 LEADER]: Becoming Leader. State: Replica: 755a97f139ec4fa2b72d000d8430386a, State: Running, Role: LEADER
I20260809 02:10:01.387982 10034 sys_catalog.cc:564] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:01.387974 10039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755a97f139ec4fa2b72d000d8430386a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42277 } }
I20260809 02:10:01.388423 10040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "755a97f139ec4fa2b72d000d8430386a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755a97f139ec4fa2b72d000d8430386a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42277 } } }
I20260809 02:10:01.388588 10040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:01.388432 10041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 755a97f139ec4fa2b72d000d8430386a. Latest consensus state: current_term: 1 leader_uuid: "755a97f139ec4fa2b72d000d8430386a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755a97f139ec4fa2b72d000d8430386a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42277 } } }
I20260809 02:10:01.388698 10041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:01.389185 10048 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:10:01.389786 10048 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:10:01.391009 10038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.384204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52798 (local address 127.5.9.190:42277)
0809 02:10:01.384455 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:01.384471 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:01.384502 (+ 31us) server_negotiation.cc:407] Connection header received
0809 02:10:01.385162 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:01.385172 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:01.385230 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:01.385417 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:01.390291 (+ 4874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:01.390296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:01.390297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:01.390304 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:01.390367 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:01.390382 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:01.390491 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:01.390736 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 02:10:01.390803 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":118,"threads_started":1}
I20260809 02:10:01.393077 10048 catalog_manager.cc:1347] Generated new cluster ID: 298e2962ec5749458baca2647260eb6b
I20260809 02:10:01.393139 10048 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:10:01.410614 10048 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:10:01.410908 10048 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:10:01.418870 10048 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a: Generated new TSK 0
I20260809 02:10:01.419305 10048 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:10:01.426424 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42277
--builtin_ntp_servers=127.5.9.148:37977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 02:10:01.551008 10058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:01.551324 10058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:01.551367 10058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:01.553190 10058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:01.553278 10058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:01.553325 10058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:10:01.555508 10058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:01.555840 10058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:01.556250 10058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:01.562091 10063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:01.562114 10064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:01.562177 10066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:01.562404 10058 server_base.cc:1034] running on GCE node
I20260809 02:10:01.562659 10058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:01.562933 10058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:01.564158 10058 hybrid_clock.cc:648] HybridClock initialized: now 1786241401564131 us; error 39 us; skew 500 ppm
I20260809 02:10:01.565717 10058 webserver.cc:460] Webserver started at http://127.5.9.129:43071/ using document root <none> and password file <none>
I20260809 02:10:01.566063 10058 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:01.566143 10058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:01.566300 10058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:01.567526 10058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "c28e6d57bf3d4ba8b03ac0e625ee0efe"
format_stamp: "Formatted at 2026-08-09 02:10:01 on dist-test-slave-p3t5"
server_key: "79322be6f56186cf0619d3c8b20c68a0"
server_key_iv: "10c256bfbc3dc376258a37049b588b8d"
server_key_version: "encryptionkey@0"
I20260809 02:10:01.567962 10058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "c28e6d57bf3d4ba8b03ac0e625ee0efe"
format_stamp: "Formatted at 2026-08-09 02:10:01 on dist-test-slave-p3t5"
server_key: "79322be6f56186cf0619d3c8b20c68a0"
server_key_iv: "10c256bfbc3dc376258a37049b588b8d"
server_key_version: "encryptionkey@0"
I20260809 02:10:01.569749 10058 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 02:10:01.570703 10072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:01.570922 10058 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:01.571030 10058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "c28e6d57bf3d4ba8b03ac0e625ee0efe"
format_stamp: "Formatted at 2026-08-09 02:10:01 on dist-test-slave-p3t5"
server_key: "79322be6f56186cf0619d3c8b20c68a0"
server_key_iv: "10c256bfbc3dc376258a37049b588b8d"
server_key_version: "encryptionkey@0"
I20260809 02:10:01.571130 10058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:01.575508 10058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:01.575836 10058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:01.575999 10058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:01.576251 10058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:01.576589 10058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:01.576644 10058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:01.576689 10058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:01.576715 10058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:01.583305 10058 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:35297
I20260809 02:10:01.583350 10189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:35297 every 8 connection(s)
I20260809 02:10:01.583747 10058 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:10:01.584133 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 10058
I20260809 02:10:01.584244 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:10:01.584471 5158 external_mini_cluster.cc:1428] Setting key 531801ccdf4bace52c33f9e29826428a
I20260809 02:10:01.585793 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42277
--builtin_ntp_servers=127.5.9.148:37977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 02:10:01.587417 10038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.584622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:40005 (local address 127.5.9.190:42277)
0809 02:10:01.584681 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:01.584683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:01.585013 (+ 330us) server_negotiation.cc:407] Connection header received
0809 02:10:01.585655 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:01.585656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:01.585663 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:01.585720 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:01.587106 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:01.587107 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:01.587108 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:01.587113 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:01.587135 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:01.587140 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:01.587209 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:01.587289 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 02:10:01.587305 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 02:10:01.587649 10192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.584631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:42277 (local address 127.5.9.129:40005)
0809 02:10:01.584857 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:01.584902 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:01.585515 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:01.585805 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:01.585809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:01.585819 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:01.586930 (+ 1111us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:10:01.586930 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:01.586964 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:10:01.586997 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:01.587216 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:01.587221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:01.587224 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:10:01.587287 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 02:10:01.587371 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":79,"threads_started":1}
I20260809 02:10:01.588246 10190 heartbeater.cc:347] Connected to a master server at 127.5.9.190:42277
I20260809 02:10:01.588364 10190 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:01.588544 10190 heartbeater.cc:511] Master 127.5.9.190:42277 requested a full tablet report, sending...
I20260809 02:10:01.589138 9998 ts_manager.cc:194] Registered new tserver with Master: c28e6d57bf3d4ba8b03ac0e625ee0efe (127.5.9.129:35297)
W20260809 02:10:01.672271 10193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:01.672510 10193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:01.672535 10193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:01.673854 10193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:01.673904 10193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:01.673954 10193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:10:01.675470 10193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:01.675755 10193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:01.676038 10193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:01.681356 10201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:01.681362 10199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:01.681605 10193 server_base.cc:1034] running on GCE node
W20260809 02:10:01.681360 10198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:01.681929 10193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:01.682226 10193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:01.683413 10193 hybrid_clock.cc:648] HybridClock initialized: now 1786241401683389 us; error 41 us; skew 500 ppm
I20260809 02:10:01.684782 10193 webserver.cc:460] Webserver started at http://127.5.9.130:41859/ using document root <none> and password file <none>
I20260809 02:10:01.685042 10193 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:01.685091 10193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:01.685256 10193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:01.686532 10193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "68e39f1560cd4a748251ab08f27d0cf9"
format_stamp: "Formatted at 2026-08-09 02:10:01 on dist-test-slave-p3t5"
server_key: "b4d751c393e0f62b1dfea70fc3413145"
server_key_iv: "70d93e2e1f94564591708b7e4b3f80ce"
server_key_version: "encryptionkey@0"
I20260809 02:10:01.686957 10193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "68e39f1560cd4a748251ab08f27d0cf9"
format_stamp: "Formatted at 2026-08-09 02:10:01 on dist-test-slave-p3t5"
server_key: "b4d751c393e0f62b1dfea70fc3413145"
server_key_iv: "70d93e2e1f94564591708b7e4b3f80ce"
server_key_version: "encryptionkey@0"
I20260809 02:10:01.688484 10193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:10:01.689450 10207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:01.689689 10193 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:01.689795 10193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "68e39f1560cd4a748251ab08f27d0cf9"
format_stamp: "Formatted at 2026-08-09 02:10:01 on dist-test-slave-p3t5"
server_key: "b4d751c393e0f62b1dfea70fc3413145"
server_key_iv: "70d93e2e1f94564591708b7e4b3f80ce"
server_key_version: "encryptionkey@0"
I20260809 02:10:01.689888 10193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:01.693495 10193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:01.693825 10193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:01.693994 10193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:01.694234 10193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:01.694581 10193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:01.694645 10193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:01.694696 10193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:01.694718 10193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:01.700789 10193 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:46781
I20260809 02:10:01.700834 10324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:46781 every 8 connection(s)
I20260809 02:10:01.701231 10193 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:10:01.702670 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 10193
I20260809 02:10:01.702782 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:10:01.702976 5158 external_mini_cluster.cc:1428] Setting key 9efd7be9b9cadc0137d48d25e96b1b6f
I20260809 02:10:01.704214 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42277
--builtin_ntp_servers=127.5.9.148:37977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 02:10:01.705044 10038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.702064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:33329 (local address 127.5.9.190:42277)
0809 02:10:01.702139 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:01.702142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:01.702391 (+ 249us) server_negotiation.cc:407] Connection header received
0809 02:10:01.703221 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:01.703223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:01.703237 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:01.703334 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:01.704751 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:01.704752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:01.704753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:01.704758 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:01.704776 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:01.704780 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:01.704829 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:01.704948 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 02:10:01.704966 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:10:01.705289 10327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.702061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:42277 (local address 127.5.9.130:33329)
0809 02:10:01.702265 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:01.702301 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:01.703056 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:01.703439 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:01.703444 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:01.703454 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:01.704574 (+ 1120us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:10:01.704575 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:01.704607 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:10:01.704641 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:01.704862 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:01.704867 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:01.704870 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:10:01.704960 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 02:10:01.705031 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260809 02:10:01.705893 10325 heartbeater.cc:347] Connected to a master server at 127.5.9.190:42277
I20260809 02:10:01.706023 10325 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:01.706194 10325 heartbeater.cc:511] Master 127.5.9.190:42277 requested a full tablet report, sending...
I20260809 02:10:01.706651 9998 ts_manager.cc:194] Registered new tserver with Master: 68e39f1560cd4a748251ab08f27d0cf9 (127.5.9.130:46781)
W20260809 02:10:01.792658 10328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:01.792897 10328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:01.792928 10328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:01.794224 10328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:01.794270 10328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:01.794297 10328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:10:01.795645 10328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:01.795869 10328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:01.796152 10328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:01.801380 10334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:01.801380 10336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:01.801491 10333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:01.801819 10328 server_base.cc:1034] running on GCE node
I20260809 02:10:01.802095 10328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:01.802366 10328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:01.803546 10328 hybrid_clock.cc:648] HybridClock initialized: now 1786241401803524 us; error 42 us; skew 500 ppm
I20260809 02:10:01.805027 10328 webserver.cc:460] Webserver started at http://127.5.9.131:36429/ using document root <none> and password file <none>
I20260809 02:10:01.805305 10328 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:01.805387 10328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:01.805555 10328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:01.806756 10328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "71a6a98c145e45398257ff255c3fe374"
format_stamp: "Formatted at 2026-08-09 02:10:01 on dist-test-slave-p3t5"
server_key: "0f7909d871f77f69f7491e73d37c800e"
server_key_iv: "00894308fe783b81ae020123689090c0"
server_key_version: "encryptionkey@0"
I20260809 02:10:01.807163 10328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "71a6a98c145e45398257ff255c3fe374"
format_stamp: "Formatted at 2026-08-09 02:10:01 on dist-test-slave-p3t5"
server_key: "0f7909d871f77f69f7491e73d37c800e"
server_key_iv: "00894308fe783b81ae020123689090c0"
server_key_version: "encryptionkey@0"
I20260809 02:10:01.808648 10328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:10:01.809478 10342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:01.809814 10328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:01.809969 10328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "71a6a98c145e45398257ff255c3fe374"
format_stamp: "Formatted at 2026-08-09 02:10:01 on dist-test-slave-p3t5"
server_key: "0f7909d871f77f69f7491e73d37c800e"
server_key_iv: "00894308fe783b81ae020123689090c0"
server_key_version: "encryptionkey@0"
I20260809 02:10:01.810058 10328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:01.813831 10328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:01.814157 10328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:01.814317 10328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:01.814550 10328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:01.814908 10328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:01.814966 10328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:01.815011 10328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:01.815039 10328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:01.821069 10328 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:36207
I20260809 02:10:01.821202 10461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:36207 every 8 connection(s)
I20260809 02:10:01.821532 10328 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:10:01.825219 10038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.822408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:58111 (local address 127.5.9.190:42277)
0809 02:10:01.822494 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:01.822496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:01.822766 (+ 270us) server_negotiation.cc:407] Connection header received
0809 02:10:01.823387 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:01.823389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:01.823396 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:01.823474 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:01.824899 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:01.824901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:01.824901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:01.824907 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:01.824927 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:01.824932 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:01.824999 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:01.825112 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 02:10:01.825132 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:10:01.825474 10464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.822388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:42277 (local address 127.5.9.131:58111)
0809 02:10:01.822627 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:01.822667 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:01.823261 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:01.823554 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:01.823564 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:01.823574 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:01.824709 (+ 1135us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:10:01.824709 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:01.824744 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:10:01.824779 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:01.825008 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:01.825015 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:01.825019 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:10:01.825124 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 02:10:01.825201 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":101,"threads_started":1}
I20260809 02:10:01.826151 10462 heartbeater.cc:347] Connected to a master server at 127.5.9.190:42277
I20260809 02:10:01.826252 10462 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:01.826448 10462 heartbeater.cc:511] Master 127.5.9.190:42277 requested a full tablet report, sending...
I20260809 02:10:01.826917 9998 ts_manager.cc:194] Registered new tserver with Master: 71a6a98c145e45398257ff255c3fe374 (127.5.9.131:36207)
I20260809 02:10:01.831126 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 10328
I20260809 02:10:01.831231 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:10:01.831446 5158 external_mini_cluster.cc:1428] Setting key 255323f25bdd5543dd633459f956aa24
I20260809 02:10:01.832525 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:10:01.835578 10038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.834100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52812 (local address 127.5.9.190:42277)
0809 02:10:01.834195 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:01.834197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:01.834336 (+ 139us) server_negotiation.cc:407] Connection header received
0809 02:10:01.834408 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:01.834409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:01.834416 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:01.834524 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:01.835286 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:01.835288 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:01.835289 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:01.835294 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:01.835311 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:01.835316 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:01.835401 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:01.835460 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 02:10:01.835485 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 02:10:01.837730 9998 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52812:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:10:01.839594 9998 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:10:01.844836 10470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.842616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:35297 (local address 127.0.0.1:48796)
0809 02:10:01.843009 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:01.843020 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:01.843054 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:01.843606 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:01.843607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:01.843609 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:01.844202 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:10:01.844203 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:01.844225 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:10:01.844258 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:01.844597 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:01.844604 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:01.844608 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:10:01.844700 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 02:10:01.844717 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":84,"threads_started":1}
I20260809 02:10:01.844949 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.842455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54358 (local address 127.5.9.130:46781)
0809 02:10:01.842678 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:01.842683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:01.842991 (+ 308us) server_negotiation.cc:407] Connection header received
0809 02:10:01.843030 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:01.843032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:01.843045 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:01.843126 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:01.844627 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:01.844634 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:01.844635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:01.844639 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:01.844687 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:01.844715 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:01.844778 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:01.844836 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 02:10:01.844853 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":133,"threads_started":1}
I20260809 02:10:01.844992 10475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.842783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48796 (local address 127.5.9.129:35297)
0809 02:10:01.843175 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:01.843180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:01.843192 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:10:01.843224 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:01.843227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:01.843237 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:01.843315 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:01.844419 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:01.844430 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:01.844431 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:01.844444 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:01.844490 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:01.844522 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:01.844588 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:01.844883 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 02:10:01.844900 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":102,"threads_started":1}
I20260809 02:10:01.845069 10473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.842492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44964 (local address 127.5.9.131:36207)
0809 02:10:01.842885 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:01.842890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:01.842902 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:10:01.842976 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:01.842979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:01.842991 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:01.843071 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:01.844527 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:01.844535 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:01.844536 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:01.844540 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:01.844600 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:01.844628 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:01.844704 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:01.844882 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 02:10:01.844947 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":100,"threads_started":1}
I20260809 02:10:01.845642 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.842382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:36207 (local address 127.0.0.1:44964)
0809 02:10:01.842784 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:01.842837 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:01.842904 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:01.843506 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:01.843512 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:01.843518 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:01.844384 (+ 866us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:10:01.844384 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:01.844395 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:10:01.844413 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:01.844712 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:01.844713 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:01.844713 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 02:10:01.844776 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 02:10:01.844800 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":125,"threads_started":1}
I20260809 02:10:01.845674 10474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:01.842352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:46781 (local address 127.0.0.1:54358)
0809 02:10:01.842783 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:01.842838 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:01.842911 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:01.843536 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:01.843537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:01.843541 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:01.844235 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:10:01.844235 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:01.844240 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:10:01.844258 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:01.844771 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:01.844772 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:01.844773 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 02:10:01.844829 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 02:10:01.844842 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":77,"threads_started":1}
I20260809 02:10:01.846500 10258 tablet_service.cc:1467] Processing CreateTablet for tablet 1bcf6b20721940fba84ec914d149d964 (DEFAULT_TABLE table=test-table [id=6fca56a99ef5446897c6d693c35897c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:01.846505 10122 tablet_service.cc:1467] Processing CreateTablet for tablet 1bcf6b20721940fba84ec914d149d964 (DEFAULT_TABLE table=test-table [id=6fca56a99ef5446897c6d693c35897c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:01.847015 10122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bcf6b20721940fba84ec914d149d964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:01.847015 10258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bcf6b20721940fba84ec914d149d964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:01.851855 10477 tablet_bootstrap.cc:492] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9: Bootstrap starting.
I20260809 02:10:01.852260 10476 tablet_bootstrap.cc:492] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe: Bootstrap starting.
I20260809 02:10:01.852497 10391 tablet_service.cc:1467] Processing CreateTablet for tablet 1bcf6b20721940fba84ec914d149d964 (DEFAULT_TABLE table=test-table [id=6fca56a99ef5446897c6d693c35897c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:01.852910 10391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bcf6b20721940fba84ec914d149d964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:01.852960 10477 tablet_bootstrap.cc:654] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:01.852959 10476 tablet_bootstrap.cc:654] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:01.853322 10476 log.cc:824] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:01.853322 10477 log.cc:824] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:01.854454 10477 tablet_bootstrap.cc:492] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9: No bootstrap required, opened a new log
I20260809 02:10:01.854561 10477 ts_tablet_manager.cc:1397] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:10:01.854820 10476 tablet_bootstrap.cc:492] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe: No bootstrap required, opened a new log
I20260809 02:10:01.854892 10476 ts_tablet_manager.cc:1397] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:10:01.857894 10480 tablet_bootstrap.cc:492] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374: Bootstrap starting.
I20260809 02:10:01.858428 10477 raft_consensus.cc:348] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e39f1560cd4a748251ab08f27d0cf9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46781 } } peers { permanent_uuid: "71a6a98c145e45398257ff255c3fe374" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 36207 } } peers { permanent_uuid: "c28e6d57bf3d4ba8b03ac0e625ee0efe" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 35297 } }
I20260809 02:10:01.858644 10477 raft_consensus.cc:374] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:01.858695 10477 raft_consensus.cc:729] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e39f1560cd4a748251ab08f27d0cf9, State: Initialized, Role: FOLLOWER
I20260809 02:10:01.858700 10476 raft_consensus.cc:348] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e39f1560cd4a748251ab08f27d0cf9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46781 } } peers { permanent_uuid: "71a6a98c145e45398257ff255c3fe374" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 36207 } } peers { permanent_uuid: "c28e6d57bf3d4ba8b03ac0e625ee0efe" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 35297 } }
I20260809 02:10:01.858872 10476 raft_consensus.cc:374] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:01.858914 10476 raft_consensus.cc:729] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c28e6d57bf3d4ba8b03ac0e625ee0efe, State: Initialized, Role: FOLLOWER
I20260809 02:10:01.858870 10477 consensus_queue.cc:260] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e39f1560cd4a748251ab08f27d0cf9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46781 } } peers { permanent_uuid: "71a6a98c145e45398257ff255c3fe374" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 36207 } } peers { permanent_uuid: "c28e6d57bf3d4ba8b03ac0e625ee0efe" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 35297 } }
I20260809 02:10:01.859030 10476 consensus_queue.cc:260] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e39f1560cd4a748251ab08f27d0cf9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46781 } } peers { permanent_uuid: "71a6a98c145e45398257ff255c3fe374" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 36207 } } peers { permanent_uuid: "c28e6d57bf3d4ba8b03ac0e625ee0efe" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 35297 } }
I20260809 02:10:01.859241 10477 ts_tablet_manager.cc:1428] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260809 02:10:01.859292 10325 heartbeater.cc:503] Master 127.5.9.190:42277 was elected leader, sending a full tablet report...
I20260809 02:10:01.859309 10476 ts_tablet_manager.cc:1428] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260809 02:10:01.859452 10190 heartbeater.cc:503] Master 127.5.9.190:42277 was elected leader, sending a full tablet report...
I20260809 02:10:01.859761 10480 tablet_bootstrap.cc:654] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:01.860162 10480 log.cc:824] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:01.861172 10480 tablet_bootstrap.cc:492] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374: No bootstrap required, opened a new log
I20260809 02:10:01.861255 10480 ts_tablet_manager.cc:1397] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:10:01.863511 10480 raft_consensus.cc:348] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e39f1560cd4a748251ab08f27d0cf9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46781 } } peers { permanent_uuid: "71a6a98c145e45398257ff255c3fe374" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 36207 } } peers { permanent_uuid: "c28e6d57bf3d4ba8b03ac0e625ee0efe" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 35297 } }
I20260809 02:10:01.863687 10480 raft_consensus.cc:374] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:01.863713 10480 raft_consensus.cc:729] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71a6a98c145e45398257ff255c3fe374, State: Initialized, Role: FOLLOWER
I20260809 02:10:01.863905 10480 consensus_queue.cc:260] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e39f1560cd4a748251ab08f27d0cf9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46781 } } peers { permanent_uuid: "71a6a98c145e45398257ff255c3fe374" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 36207 } } peers { permanent_uuid: "c28e6d57bf3d4ba8b03ac0e625ee0efe" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 35297 } }
I20260809 02:10:01.864283 10480 ts_tablet_manager.cc:1428] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:10:01.864295 10462 heartbeater.cc:503] Master 127.5.9.190:42277 was elected leader, sending a full tablet report...
W20260809 02:10:01.951881 10326 tablet.cc:2369] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:10:02.072139 10463 tablet.cc:2369] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:10:02.084626 10191 tablet.cc:2369] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:10:02.267081 10486 raft_consensus.cc:482] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:10:02.267235 10486 raft_consensus.cc:504] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e39f1560cd4a748251ab08f27d0cf9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46781 } } peers { permanent_uuid: "71a6a98c145e45398257ff255c3fe374" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 36207 } } peers { permanent_uuid: "c28e6d57bf3d4ba8b03ac0e625ee0efe" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 35297 } }
I20260809 02:10:02.267730 10486 leader_election.cc:288] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e39f1560cd4a748251ab08f27d0cf9 (127.5.9.130:46781), c28e6d57bf3d4ba8b03ac0e625ee0efe (127.5.9.129:35297)
I20260809 02:10:02.269302 10464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:02.267956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:35297 (local address 127.5.9.131:41345)
0809 02:10:02.268019 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:02.268031 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:02.268076 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:02.268262 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:02.268263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:02.268265 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:02.268797 (+ 532us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:10:02.268798 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:02.268806 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:10:02.268822 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:02.269141 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:02.269143 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:02.269143 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 02:10:02.269207 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 02:10:02.269221 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 02:10:02.269352 10475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:02.268020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:41345 (local address 127.5.9.129:35297)
0809 02:10:02.268073 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:02.268075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:02.268088 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:10:02.268150 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:02.268151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:02.268158 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:02.268231 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:02.269038 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:02.269040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:02.269040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:02.269045 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:02.269065 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:02.269069 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:02.269130 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:02.269238 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 02:10:02.269255 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 02:10:02.269446 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:02.267968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:46781 (local address 127.5.9.131:46341)
0809 02:10:02.268160 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:02.268171 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:02.268222 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:02.268371 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:02.268372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:02.268404 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:02.269027 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:10:02.269027 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:02.269034 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:10:02.269048 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:02.269277 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:02.269278 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:02.269278 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 02:10:02.269349 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 02:10:02.269362 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":118,"threads_started":1}
I20260809 02:10:02.269445 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:02.268055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:46341 (local address 127.5.9.130:46781)
0809 02:10:02.268099 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:02.268102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:02.268239 (+ 137us) server_negotiation.cc:407] Connection header received
0809 02:10:02.268270 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:02.268271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:02.268277 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:02.268342 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:02.269148 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:02.269149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:02.269149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:02.269154 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:02.269173 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:02.269178 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:02.269252 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:02.269349 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 02:10:02.269363 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:10:02.269745 10144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bcf6b20721940fba84ec914d149d964" candidate_uuid: "71a6a98c145e45398257ff255c3fe374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c28e6d57bf3d4ba8b03ac0e625ee0efe" is_pre_election: true
I20260809 02:10:02.269958 10144 raft_consensus.cc:2454] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71a6a98c145e45398257ff255c3fe374 in term 0.
I20260809 02:10:02.269922 10279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bcf6b20721940fba84ec914d149d964" candidate_uuid: "71a6a98c145e45398257ff255c3fe374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e39f1560cd4a748251ab08f27d0cf9" is_pre_election: true
I20260809 02:10:02.270105 10279 raft_consensus.cc:2454] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71a6a98c145e45398257ff255c3fe374 in term 0.
I20260809 02:10:02.270228 10345 leader_election.cc:302] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71a6a98c145e45398257ff255c3fe374, c28e6d57bf3d4ba8b03ac0e625ee0efe; no voters:
I20260809 02:10:02.270378 10486 raft_consensus.cc:2781] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:10:02.270439 10486 raft_consensus.cc:482] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:10:02.270463 10486 raft_consensus.cc:3037] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:02.271235 10486 raft_consensus.cc:504] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e39f1560cd4a748251ab08f27d0cf9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46781 } } peers { permanent_uuid: "71a6a98c145e45398257ff255c3fe374" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 36207 } } peers { permanent_uuid: "c28e6d57bf3d4ba8b03ac0e625ee0efe" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 35297 } }
I20260809 02:10:02.271399 10486 leader_election.cc:288] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [CANDIDATE]: Term 1 election: Requested vote from peers 68e39f1560cd4a748251ab08f27d0cf9 (127.5.9.130:46781), c28e6d57bf3d4ba8b03ac0e625ee0efe (127.5.9.129:35297)
I20260809 02:10:02.271587 10144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bcf6b20721940fba84ec914d149d964" candidate_uuid: "71a6a98c145e45398257ff255c3fe374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c28e6d57bf3d4ba8b03ac0e625ee0efe"
I20260809 02:10:02.271587 10279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bcf6b20721940fba84ec914d149d964" candidate_uuid: "71a6a98c145e45398257ff255c3fe374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e39f1560cd4a748251ab08f27d0cf9"
I20260809 02:10:02.271677 10279 raft_consensus.cc:3037] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:02.271677 10144 raft_consensus.cc:3037] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:02.272651 10144 raft_consensus.cc:2454] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71a6a98c145e45398257ff255c3fe374 in term 1.
I20260809 02:10:02.272651 10279 raft_consensus.cc:2454] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71a6a98c145e45398257ff255c3fe374 in term 1.
I20260809 02:10:02.272841 10345 leader_election.cc:302] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71a6a98c145e45398257ff255c3fe374, c28e6d57bf3d4ba8b03ac0e625ee0efe; no voters:
I20260809 02:10:02.272953 10486 raft_consensus.cc:2781] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:02.273100 10486 raft_consensus.cc:686] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [term 1 LEADER]: Becoming Leader. State: Replica: 71a6a98c145e45398257ff255c3fe374, State: Running, Role: LEADER
I20260809 02:10:02.273207 10486 consensus_queue.cc:237] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e39f1560cd4a748251ab08f27d0cf9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46781 } } peers { permanent_uuid: "71a6a98c145e45398257ff255c3fe374" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 36207 } } peers { permanent_uuid: "c28e6d57bf3d4ba8b03ac0e625ee0efe" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 35297 } }
I20260809 02:10:02.274150 9998 catalog_manager.cc:5495] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71a6a98c145e45398257ff255c3fe374 (127.5.9.131). New cstate: current_term: 1 leader_uuid: "71a6a98c145e45398257ff255c3fe374" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e39f1560cd4a748251ab08f27d0cf9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71a6a98c145e45398257ff255c3fe374" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 36207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c28e6d57bf3d4ba8b03ac0e625ee0efe" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 35297 } health_report { overall_health: UNKNOWN } } }
I20260809 02:10:02.288797 10473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:02.287574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44966 (local address 127.5.9.131:36207)
0809 02:10:02.287646 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:02.287648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:02.287660 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:10:02.287727 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:02.287728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:02.287741 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:02.287799 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:02.288530 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:02.288531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:02.288531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:02.288536 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:02.288560 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:02.288565 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:02.288632 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:02.288690 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 02:10:02.288705 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:10:02.291015 10279 raft_consensus.cc:1261] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9 [term 1 FOLLOWER]: Refusing update from remote peer 71a6a98c145e45398257ff255c3fe374: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:02.291388 10144 raft_consensus.cc:1261] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe [term 1 FOLLOWER]: Refusing update from remote peer 71a6a98c145e45398257ff255c3fe374: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:02.291406 10488 consensus_queue.cc:1035] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e39f1560cd4a748251ab08f27d0cf9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:02.291832 10488 consensus_queue.cc:1035] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c28e6d57bf3d4ba8b03ac0e625ee0efe" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 35297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:02.294377 10493 mvcc.cc:205] Tried to move back new op lower bound from 7316444783780843520 to 7316444783711719424. Current Snapshot: MvccSnapshot[applied={T|T < 7316444783780843520}]
I20260809 02:10:02.296861 9998 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:02.296984 9998 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:02.297868 9998 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 755a97f139ec4fa2b72d000d8430386a: Sending DeleteTablet for 3 replicas of tablet 1bcf6b20721940fba84ec914d149d964
I20260809 02:10:02.298338 10122 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bcf6b20721940fba84ec914d149d964 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:02 UTC) from {username='slave'} at 127.0.0.1:48796
I20260809 02:10:02.298338 10391 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bcf6b20721940fba84ec914d149d964 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:02 UTC) from {username='slave'} at 127.0.0.1:44964
I20260809 02:10:02.298391 10258 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bcf6b20721940fba84ec914d149d964 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:02 UTC) from {username='slave'} at 127.0.0.1:54358
I20260809 02:10:02.298574 10502 tablet_replica.cc:330] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe: stopping tablet replica
I20260809 02:10:02.298684 10502 raft_consensus.cc:2229] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:02.298782 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 10058
I20260809 02:10:02.298820 10502 raft_consensus.cc:2258] T 1bcf6b20721940fba84ec914d149d964 P c28e6d57bf3d4ba8b03ac0e625ee0efe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:02.305845 10504 tablet_replica.cc:330] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9: stopping tablet replica
I20260809 02:10:02.306048 10504 raft_consensus.cc:2229] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:02.306197 10504 raft_consensus.cc:2258] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:02.306216 10503 tablet_replica.cc:330] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374: stopping tablet replica
I20260809 02:10:02.306418 10503 raft_consensus.cc:2229] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:10:02.306610 10503 raft_consensus.cc:2258] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:02.307089 10504 ts_tablet_manager.cc:1905] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:02.308435 10503 ts_tablet_manager.cc:1905] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:02.309478 10504 ts_tablet_manager.cc:1918] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:02.309578 10504 log.cc:1196] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/1bcf6b20721940fba84ec914d149d964
I20260809 02:10:02.309897 10504 ts_tablet_manager.cc:1939] T 1bcf6b20721940fba84ec914d149d964 P 68e39f1560cd4a748251ab08f27d0cf9: Deleting consensus metadata
I20260809 02:10:02.310237 10503 ts_tablet_manager.cc:1918] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:02.310338 10503 log.cc:1196] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/1bcf6b20721940fba84ec914d149d964
I20260809 02:10:02.310608 10503 ts_tablet_manager.cc:1939] T 1bcf6b20721940fba84ec914d149d964 P 71a6a98c145e45398257ff255c3fe374: Deleting consensus metadata
I20260809 02:10:02.310563 9985 catalog_manager.cc:4841] TS 68e39f1560cd4a748251ab08f27d0cf9 (127.5.9.130:46781): tablet 1bcf6b20721940fba84ec914d149d964 (table test-table [id=6fca56a99ef5446897c6d693c35897c1]) successfully deleted
I20260809 02:10:02.311106 9983 catalog_manager.cc:4841] TS 71a6a98c145e45398257ff255c3fe374 (127.5.9.131:36207): tablet 1bcf6b20721940fba84ec914d149d964 (table test-table [id=6fca56a99ef5446897c6d693c35897c1]) successfully deleted
W20260809 02:10:02.313215 9983 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:35297 (error 108)
I20260809 02:10:02.313444 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 10193
I20260809 02:10:02.313890 10474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:02.313728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:35297 (local address 127.0.0.1:48806)
0809 02:10:02.313811 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:02.313851 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:35297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 02:10:02.314819 9983 catalog_manager.cc:4584] TS c28e6d57bf3d4ba8b03ac0e625ee0efe (127.5.9.129:35297): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1bcf6b20721940fba84ec914d149d964: Network error: Client connection negotiation failed: client connection to 127.5.9.129:35297: connect: Connection refused (error 111)
I20260809 02:10:02.320904 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 10328
I20260809 02:10:02.327993 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 9966
2026-08-09T02:10:02Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1101 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 02:10:02.350847 5158 mini_oidc.cc:152] Starting JWKS server
I20260809 02:10:02.352310 5158 webserver.cc:460] Webserver started at https://127.0.0.1:38271/ using document root <none> and password file <none>
I20260809 02:10:02.352475 5158 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 02:10:02.352735 5158 webserver.cc:460] Webserver started at http://127.0.0.1:37917/ using document root <none> and password file <none>
2026-08-09T02:10:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:10:02Z Disabled control of system clock
I20260809 02:10:02.363344 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:34257
--webserver_interface=127.5.9.190
--webserver_port=0
--builtin_ntp_servers=127.5.9.148:41107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:34257
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38271/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/testchainca.cert with env {}
W20260809 02:10:02.453461 10512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:02.453800 10512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:02.453832 10512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:02.455060 10512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:02.455102 10512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:02.455116 10512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:02.455128 10512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:02.455140 10512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 02:10:02.455153 10512 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 02:10:02.455164 10512 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38271/jwks.json
I20260809 02:10:02.456784 10512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:34257
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:34257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38271/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:02.457034 10512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:02.457222 10512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:02.462452 10517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:02.462456 10520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:02.462519 10518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:02.462755 10512 server_base.cc:1034] running on GCE node
I20260809 02:10:02.463043 10512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:02.463382 10512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:02.464557 10512 hybrid_clock.cc:648] HybridClock initialized: now 1786241402464546 us; error 59 us; skew 500 ppm
I20260809 02:10:02.466158 10512 webserver.cc:460] Webserver started at http://127.5.9.190:36799/ using document root <none> and password file <none>
I20260809 02:10:02.466450 10512 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:02.466526 10512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:02.466683 10512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:02.467815 10512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "9d40323c051c4e388a34984380932075"
format_stamp: "Formatted at 2026-08-09 02:10:02 on dist-test-slave-p3t5"
server_key: "c8cbfbbd305250ad40147a6aecba798f"
server_key_iv: "e4bf8471d81b091fe6b423a415b579d0"
server_key_version: "encryptionkey@0"
I20260809 02:10:02.468282 10512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "9d40323c051c4e388a34984380932075"
format_stamp: "Formatted at 2026-08-09 02:10:02 on dist-test-slave-p3t5"
server_key: "c8cbfbbd305250ad40147a6aecba798f"
server_key_iv: "e4bf8471d81b091fe6b423a415b579d0"
server_key_version: "encryptionkey@0"
I20260809 02:10:02.469741 10512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260809 02:10:02.470556 10526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:02.470743 10512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:02.470820 10512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "9d40323c051c4e388a34984380932075"
format_stamp: "Formatted at 2026-08-09 02:10:02 on dist-test-slave-p3t5"
server_key: "c8cbfbbd305250ad40147a6aecba798f"
server_key_iv: "e4bf8471d81b091fe6b423a415b579d0"
server_key_version: "encryptionkey@0"
I20260809 02:10:02.470887 10512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:02.496033 10512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:02.496366 10512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:02.496481 10512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:02.501147 10512 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:34257
I20260809 02:10:02.501240 10582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:34257 every 8 connection(s)
I20260809 02:10:02.501608 10512 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:10:02.502892 10583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:02.505350 10583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075: Bootstrap starting.
I20260809 02:10:02.506069 10583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:02.506335 10583 log.cc:824] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:02.507159 10583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075: No bootstrap required, opened a new log
I20260809 02:10:02.508992 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 10512
I20260809 02:10:02.508884 10583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d40323c051c4e388a34984380932075" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34257 } }
I20260809 02:10:02.509111 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:10:02.509109 10583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:02.509160 10583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d40323c051c4e388a34984380932075, State: Initialized, Role: FOLLOWER
I20260809 02:10:02.509313 5158 external_mini_cluster.cc:1428] Setting key e2e1d1971a787a876a3e5040c69053a5
I20260809 02:10:02.509341 10583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d40323c051c4e388a34984380932075" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34257 } }
I20260809 02:10:02.509435 10583 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:10:02.509477 10583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:10:02.509541 10583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:02.510427 10583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d40323c051c4e388a34984380932075" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34257 } }
I20260809 02:10:02.510567 10583 leader_election.cc:302] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d40323c051c4e388a34984380932075; no voters:
I20260809 02:10:02.510758 10583 leader_election.cc:288] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:10:02.510831 10588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:02.510990 10588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [term 1 LEADER]: Becoming Leader. State: Replica: 9d40323c051c4e388a34984380932075, State: Running, Role: LEADER
I20260809 02:10:02.511083 10583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:02.511081 10588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d40323c051c4e388a34984380932075" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34257 } }
I20260809 02:10:02.512101 10590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d40323c051c4e388a34984380932075. Latest consensus state: current_term: 1 leader_uuid: "9d40323c051c4e388a34984380932075" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d40323c051c4e388a34984380932075" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34257 } } }
I20260809 02:10:02.512192 10590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:02.512625 10589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d40323c051c4e388a34984380932075" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d40323c051c4e388a34984380932075" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34257 } } }
I20260809 02:10:02.512696 10589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:02.512833 10597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:10:02.513557 10597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:10:02.515568 10597 catalog_manager.cc:1347] Generated new cluster ID: 9edf9d0d24a7424a82b762e049a88ef8
I20260809 02:10:02.515628 10597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:10:02.516692 10587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:02.509950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43750 (local address 127.5.9.190:34257)
0809 02:10:02.510183 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:02.510191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:02.510225 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:10:02.510344 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:02.510350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:02.510417 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:02.510589 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:02.510901 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:02.513031 (+ 2130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:02.513999 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:02.514578 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:02.514699 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:02.515738 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:02.515742 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:02.515742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:02.515987 (+ 245us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:02.516029 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:02.516044 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:02.516099 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:02.516378 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 02:10:02.516463 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":108,"threads_started":1}
I20260809 02:10:02.517599 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:10:02.529158 10597 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:10:02.530035 10597 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:10:02.534789 10597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d40323c051c4e388a34984380932075: Generated new TSK 0
I20260809 02:10:02.535030 10597 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:10:02.565676 10587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:02.541883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43756 (local address 127.5.9.190:34257)
0809 02:10:02.541955 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:02.541957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:02.542090 (+ 133us) server_negotiation.cc:407] Connection header received
0809 02:10:02.542230 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:02.542231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:02.542244 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:02.542310 (+ 66us) server_negotiation.cc:226] Negotiated authn=JWT
0809 02:10:02.542495 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:02.543031 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:02.543885 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:02.544102 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:02.544180 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:02.544432 (+ 252us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 02:10:02.565370 (+ 20938us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 02:10:02.565567 (+ 197us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43756: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 02:10:02.565761 10587 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43756: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
E20260809 02:10:02.566174 10613 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 02:10:02.566303 10613 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260809 02:10:02.566371 10612 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:02.541794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:34257 (local address 127.0.0.1:43756)
0809 02:10:02.542003 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:02.542014 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:02.542114 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:02.542320 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:02.542320 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:02.542322 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 02:10:02.542416 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:02.542418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:02.543134 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:02.543135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:02.543785 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:02.543786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:02.544179 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:02.544184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:02.544264 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:02.544287 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 02:10:02.565561 (+ 21274us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 02:10:02.565892 (+ 331us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.9.190:34257: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":183,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.5.9.190:34257: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38271/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 02:10:02.567562 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 10512
2026-08-09T02:10:02Z chronyd exiting
I20260809 02:10:02.607460 5158 test_util.cc:175] -----------------------------------------------
I20260809 02:10:02.607577 5158 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241377393328-5158-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (258 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 02:10:02.608448 5158 mini_oidc.cc:152] Starting JWKS server
I20260809 02:10:02.609182 5158 webserver.cc:460] Webserver started at https://127.0.0.1:44645/ using document root <none> and password file <none>
I20260809 02:10:02.609264 5158 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 02:10:02.609458 5158 webserver.cc:460] Webserver started at http://127.0.0.1:45773/ using document root <none> and password file <none>
2026-08-09T02:10:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:10:02Z Disabled control of system clock
I20260809 02:10:02.619560 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:40751
--webserver_interface=127.5.9.190
--webserver_port=0
--builtin_ntp_servers=127.5.9.148:37809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:40751
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44645/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/testchainca.cert with env {}
W20260809 02:10:02.706622 10620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:02.706913 10620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:02.706939 10620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:02.708266 10620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:02.708320 10620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:02.708333 10620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:02.708348 10620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:02.708359 10620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 02:10:02.708372 10620 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 02:10:02.708384 10620 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44645/jwks.json
I20260809 02:10:02.710635 10620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:40751
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:40751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44645/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:02.710987 10620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:02.711246 10620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:02.716519 10627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:02.716539 10629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:02.716539 10625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:02.716851 10620 server_base.cc:1034] running on GCE node
I20260809 02:10:02.717126 10620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:02.717456 10620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:02.718636 10620 hybrid_clock.cc:648] HybridClock initialized: now 1786241402718604 us; error 48 us; skew 500 ppm
I20260809 02:10:02.720002 10620 webserver.cc:460] Webserver started at http://127.5.9.190:41569/ using document root <none> and password file <none>
I20260809 02:10:02.720273 10620 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:02.720348 10620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:02.720508 10620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:02.721724 10620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "7a3b4b9cbf5541e7bbd1363be1805801"
format_stamp: "Formatted at 2026-08-09 02:10:02 on dist-test-slave-p3t5"
server_key: "a569e5fd6e015bbd69bdfd48846b096b"
server_key_iv: "5b7c472b5a0e74c22fd54aa587091c49"
server_key_version: "encryptionkey@0"
I20260809 02:10:02.722122 10620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "7a3b4b9cbf5541e7bbd1363be1805801"
format_stamp: "Formatted at 2026-08-09 02:10:02 on dist-test-slave-p3t5"
server_key: "a569e5fd6e015bbd69bdfd48846b096b"
server_key_iv: "5b7c472b5a0e74c22fd54aa587091c49"
server_key_version: "encryptionkey@0"
I20260809 02:10:02.723655 10620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:10:02.724498 10635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:02.724742 10620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:10:02.724870 10620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "7a3b4b9cbf5541e7bbd1363be1805801"
format_stamp: "Formatted at 2026-08-09 02:10:02 on dist-test-slave-p3t5"
server_key: "a569e5fd6e015bbd69bdfd48846b096b"
server_key_iv: "5b7c472b5a0e74c22fd54aa587091c49"
server_key_version: "encryptionkey@0"
I20260809 02:10:02.724961 10620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:02.759629 10620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:02.760028 10620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:02.760210 10620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:02.764740 10620 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:40751
I20260809 02:10:02.764783 10689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:40751 every 8 connection(s)
I20260809 02:10:02.765166 10620 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:10:02.765512 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 10620
I20260809 02:10:02.765651 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:10:02.765832 5158 external_mini_cluster.cc:1428] Setting key 8f43cfd7442b71974397d762ae412341
I20260809 02:10:02.766571 10690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:02.769851 10690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801: Bootstrap starting.
I20260809 02:10:02.770633 10690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:02.771003 10690 log.cc:824] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:02.772023 10690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801: No bootstrap required, opened a new log
I20260809 02:10:02.772045 10693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:02.766409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43190 (local address 127.5.9.190:40751)
0809 02:10:02.766659 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:02.766670 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:02.766709 (+ 39us) server_negotiation.cc:407] Connection header received
0809 02:10:02.767173 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:02.767180 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:02.767245 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:02.767372 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:02.767644 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:02.768561 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:02.769718 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:02.770208 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:02.770305 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:02.771214 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:02.771218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:02.771219 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:02.771407 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:02.771443 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:02.771455 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:02.771515 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:02.771697 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 02:10:02.771791 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":112,"threads_started":1}
I20260809 02:10:02.773475 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:10:02.774887 10690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b4b9cbf5541e7bbd1363be1805801" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 40751 } }
I20260809 02:10:02.775048 10690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:02.775071 10690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a3b4b9cbf5541e7bbd1363be1805801, State: Initialized, Role: FOLLOWER
I20260809 02:10:02.775223 10690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b4b9cbf5541e7bbd1363be1805801" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 40751 } }
I20260809 02:10:02.775305 10690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:10:02.775359 10690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:10:02.775422 10690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:02.776324 10690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b4b9cbf5541e7bbd1363be1805801" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 40751 } }
I20260809 02:10:02.776454 10690 leader_election.cc:302] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a3b4b9cbf5541e7bbd1363be1805801; no voters:
I20260809 02:10:02.776690 10690 leader_election.cc:288] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:10:02.776789 10695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:02.776949 10695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [term 1 LEADER]: Becoming Leader. State: Replica: 7a3b4b9cbf5541e7bbd1363be1805801, State: Running, Role: LEADER
I20260809 02:10:02.776986 10690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:02.777057 10695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b4b9cbf5541e7bbd1363be1805801" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 40751 } }
I20260809 02:10:02.777494 10696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a3b4b9cbf5541e7bbd1363be1805801" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b4b9cbf5541e7bbd1363be1805801" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 40751 } } }
I20260809 02:10:02.777535 10697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a3b4b9cbf5541e7bbd1363be1805801. Latest consensus state: current_term: 1 leader_uuid: "7a3b4b9cbf5541e7bbd1363be1805801" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b4b9cbf5541e7bbd1363be1805801" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 40751 } } }
I20260809 02:10:02.777729 10697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:02.777967 10696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:02.778143 10704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:10:02.778653 10704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:10:02.780256 10704 catalog_manager.cc:1347] Generated new cluster ID: 0642e6886f4e484b99c7f9f928bf6bfa
I20260809 02:10:02.780313 10704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:10:02.789666 10704 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:10:02.790262 10704 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:10:02.795615 10704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a3b4b9cbf5541e7bbd1363be1805801: Generated new TSK 0
I20260809 02:10:02.795856 10704 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 02:10:02.819849 10720 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 02:10:02.819959 10720 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 02:10:02.820406 10693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:02.805736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43204 (local address 127.5.9.190:40751)
0809 02:10:02.805794 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:02.805796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:02.805945 (+ 149us) server_negotiation.cc:407] Connection header received
0809 02:10:02.806000 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:02.806001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:02.806022 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:02.806109 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0809 02:10:02.806276 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:02.806834 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:02.807685 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:02.807900 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:02.807984 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:02.808212 (+ 228us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 02:10:02.820115 (+ 11903us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44645/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 02:10:02.820320 (+ 205us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43204: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44645/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 02:10:02.820477 10693 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43204: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44645/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 02:10:02.820498 10719 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:02.805626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:40751 (local address 127.0.0.1:43204)
0809 02:10:02.805819 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:02.805831 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:02.805907 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:02.806111 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:02.806112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:02.806114 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 02:10:02.806182 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:02.806184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:02.806941 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:02.806943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:02.807585 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:02.807587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:02.807982 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:02.807986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:02.808061 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:02.808075 (+ 14us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 02:10:02.820255 (+ 12180us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44645/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 02:10:02.820384 (+ 129us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.9.190:40751: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44645/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":135,"threads_started":1}
I20260809 02:10:02.820926 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 10620
2026-08-09T02:10:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (252 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 02:10:02.861485 5158 mini_oidc.cc:152] Starting JWKS server
I20260809 02:10:02.862282 5158 webserver.cc:460] Webserver started at https://127.0.0.1:41179/ using document root <none> and password file <none>
I20260809 02:10:02.862367 5158 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 02:10:02.862528 5158 webserver.cc:460] Webserver started at http://127.0.0.1:33261/ using document root <none> and password file <none>
2026-08-09T02:10:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:10:02Z Disabled control of system clock
I20260809 02:10:02.873255 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:39445
--webserver_interface=127.5.9.190
--webserver_port=0
--builtin_ntp_servers=127.5.9.148:44351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:39445
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41179/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 02:10:02.962038 10726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:02.962281 10726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:02.962307 10726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:02.963505 10726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:02.963546 10726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:02.963559 10726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:02.963572 10726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:02.963584 10726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 02:10:02.963596 10726 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 02:10:02.963608 10726 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41179/jwks.json
I20260809 02:10:02.964982 10726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:44351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:39445
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:39445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41179/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:02.965216 10726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:02.965487 10726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:02.970890 10731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:02.970934 10732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:02.971096 10726 server_base.cc:1034] running on GCE node
W20260809 02:10:02.971269 10735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:02.971576 10726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:02.971967 10726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:02.973169 10726 hybrid_clock.cc:648] HybridClock initialized: now 1786241402973146 us; error 46 us; skew 500 ppm
I20260809 02:10:02.974862 10726 webserver.cc:460] Webserver started at http://127.5.9.190:33355/ using document root <none> and password file <none>
I20260809 02:10:02.975150 10726 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:02.975229 10726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:02.975404 10726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:02.976626 10726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "098c2fef76db457eb66b63b938089416"
format_stamp: "Formatted at 2026-08-09 02:10:02 on dist-test-slave-p3t5"
server_key: "6f4b188a448bd00253a102e02894ce9c"
server_key_iv: "ff0b9a40b9d943c74f256b93cb9fdd1f"
server_key_version: "encryptionkey@0"
I20260809 02:10:02.977035 10726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "098c2fef76db457eb66b63b938089416"
format_stamp: "Formatted at 2026-08-09 02:10:02 on dist-test-slave-p3t5"
server_key: "6f4b188a448bd00253a102e02894ce9c"
server_key_iv: "ff0b9a40b9d943c74f256b93cb9fdd1f"
server_key_version: "encryptionkey@0"
I20260809 02:10:02.978636 10726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:10:02.979523 10741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:02.979750 10726 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:10:02.979874 10726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "098c2fef76db457eb66b63b938089416"
format_stamp: "Formatted at 2026-08-09 02:10:02 on dist-test-slave-p3t5"
server_key: "6f4b188a448bd00253a102e02894ce9c"
server_key_iv: "ff0b9a40b9d943c74f256b93cb9fdd1f"
server_key_version: "encryptionkey@0"
I20260809 02:10:02.979974 10726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:02.999154 10726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:02.999536 10726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:02.999706 10726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:03.005039 10726 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:39445
I20260809 02:10:03.005082 10798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:39445 every 8 connection(s)
I20260809 02:10:03.005470 10726 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:10:03.006541 10799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:03.009213 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 10726
I20260809 02:10:03.009325 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:10:03.009490 10799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416: Bootstrap starting.
I20260809 02:10:03.009545 5158 external_mini_cluster.cc:1428] Setting key 456132a06ea1fa28798b28ca02bee4b6
I20260809 02:10:03.010337 10799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:03.010669 10799 log.cc:824] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:03.011498 10799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416: No bootstrap required, opened a new log
I20260809 02:10:03.013396 10799 raft_consensus.cc:348] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c2fef76db457eb66b63b938089416" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39445 } }
I20260809 02:10:03.013587 10799 raft_consensus.cc:374] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:03.013670 10799 raft_consensus.cc:729] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098c2fef76db457eb66b63b938089416, State: Initialized, Role: FOLLOWER
I20260809 02:10:03.013823 10799 consensus_queue.cc:260] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c2fef76db457eb66b63b938089416" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39445 } }
I20260809 02:10:03.013936 10799 raft_consensus.cc:388] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:10:03.013983 10799 raft_consensus.cc:482] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:10:03.014039 10799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:03.014905 10799 raft_consensus.cc:504] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c2fef76db457eb66b63b938089416" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39445 } }
I20260809 02:10:03.015008 10799 leader_election.cc:302] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 098c2fef76db457eb66b63b938089416; no voters:
I20260809 02:10:03.015228 10799 leader_election.cc:288] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:10:03.015326 10804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:03.015542 10804 raft_consensus.cc:686] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [term 1 LEADER]: Becoming Leader. State: Replica: 098c2fef76db457eb66b63b938089416, State: Running, Role: LEADER
I20260809 02:10:03.015578 10799 sys_catalog.cc:564] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:03.015653 10804 consensus_queue.cc:237] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c2fef76db457eb66b63b938089416" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39445 } }
I20260809 02:10:03.015796 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:03.010169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47558 (local address 127.5.9.190:39445)
0809 02:10:03.010429 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:03.010441 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:03.010476 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:10:03.011027 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:03.011036 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:03.011107 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:03.011282 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:03.011477 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:03.012484 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:03.013353 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:03.013967 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:03.014101 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:03.014808 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:03.014812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:03.014812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:03.015049 (+ 237us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:03.015120 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:03.015137 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:03.015199 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:03.015290 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 02:10:03.015529 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":117,"threads_started":1}
I20260809 02:10:03.016058 10804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 098c2fef76db457eb66b63b938089416. Latest consensus state: current_term: 1 leader_uuid: "098c2fef76db457eb66b63b938089416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c2fef76db457eb66b63b938089416" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39445 } } }
I20260809 02:10:03.016084 10805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "098c2fef76db457eb66b63b938089416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c2fef76db457eb66b63b938089416" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39445 } } }
I20260809 02:10:03.016145 10804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:03.016173 10805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:03.016381 10812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:10:03.017012 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:10:03.017107 10812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:10:03.018882 10812 catalog_manager.cc:1347] Generated new cluster ID: 14413e7057154964b0ce3b6eeb00df85
I20260809 02:10:03.018940 10812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:10:03.027338 10812 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:10:03.028096 10812 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:10:03.032444 10812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 098c2fef76db457eb66b63b938089416: Generated new TSK 0
I20260809 02:10:03.032701 10812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:10:03.052069 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:03.047943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47568 (local address 127.5.9.190:39445)
0809 02:10:03.048017 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:03.048019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:03.048153 (+ 134us) server_negotiation.cc:407] Connection header received
0809 02:10:03.048243 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:03.048244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:03.048255 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:03.048322 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0809 02:10:03.048480 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:03.049020 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:03.050210 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:03.050417 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:03.050475 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:03.050721 (+ 246us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 02:10:03.051798 (+ 1077us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41179/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 02:10:03.051941 (+ 143us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47568: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41179/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 02:10:03.052148 10802 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47568: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41179/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 02:10:03.052531 10830 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:03.047880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39445 (local address 127.0.0.1:47568)
0809 02:10:03.048075 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:03.048087 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:03.048159 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:03.048339 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:03.048340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:03.048342 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 02:10:03.048412 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:03.048415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:03.049110 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:03.049111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:03.050124 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:03.050127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:03.050495 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:03.050499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:03.050565 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:03.050571 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 02:10:03.051919 (+ 1348us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41179/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 02:10:03.052036 (+ 117us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.9.190:39445: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41179/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":122,"threads_started":1}
I20260809 02:10:03.052783 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 10726
E20260809 02:10:03.065058 10831 webserver.cc:533] Webserver: sslize failed:
E20260809 02:10:03.065182 10831 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T02:10:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (232 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T02:10:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:10:03Z Disabled control of system clock
I20260809 02:10:03.103801 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:33367
--webserver_interface=127.5.9.190
--webserver_port=0
--builtin_ntp_servers=127.5.9.148:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:33367
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/testchainca.cert with env {}
W20260809 02:10:03.188753 10835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:03.189047 10835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:03.189078 10835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:03.190311 10835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:03.190351 10835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:03.190366 10835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:03.190377 10835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:03.190390 10835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 02:10:03.190402 10835 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 02:10:03.190414 10835 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 02:10:03.191991 10835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:33367
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:33367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:03.192227 10835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:03.192415 10835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:03.197708 10841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:03.197801 10844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:03.197708 10840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:03.198088 10835 server_base.cc:1034] running on GCE node
I20260809 02:10:03.198402 10835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:03.198738 10835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:03.199905 10835 hybrid_clock.cc:648] HybridClock initialized: now 1786241403199890 us; error 34 us; skew 500 ppm
I20260809 02:10:03.201269 10835 webserver.cc:460] Webserver started at http://127.5.9.190:43317/ using document root <none> and password file <none>
I20260809 02:10:03.201545 10835 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:03.201596 10835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:03.201781 10835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:03.202983 10835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "076d825dadfd4417a104a0167c76ef80"
format_stamp: "Formatted at 2026-08-09 02:10:03 on dist-test-slave-p3t5"
server_key: "d2880c1534d06aabac019db1fecda7b3"
server_key_iv: "28186502e2e3998756a86b401e329cb6"
server_key_version: "encryptionkey@0"
I20260809 02:10:03.203393 10835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "076d825dadfd4417a104a0167c76ef80"
format_stamp: "Formatted at 2026-08-09 02:10:03 on dist-test-slave-p3t5"
server_key: "d2880c1534d06aabac019db1fecda7b3"
server_key_iv: "28186502e2e3998756a86b401e329cb6"
server_key_version: "encryptionkey@0"
I20260809 02:10:03.204850 10835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:10:03.205816 10850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:03.206151 10835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:10:03.206293 10835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "076d825dadfd4417a104a0167c76ef80"
format_stamp: "Formatted at 2026-08-09 02:10:03 on dist-test-slave-p3t5"
server_key: "d2880c1534d06aabac019db1fecda7b3"
server_key_iv: "28186502e2e3998756a86b401e329cb6"
server_key_version: "encryptionkey@0"
I20260809 02:10:03.206391 10835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:03.225307 10835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:03.225762 10835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:03.225946 10835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:03.230255 10835 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:33367
I20260809 02:10:03.230300 10903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:33367 every 8 connection(s)
I20260809 02:10:03.230697 10835 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:10:03.231926 10904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:03.234505 10904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80: Bootstrap starting.
I20260809 02:10:03.235186 10904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:03.235531 10904 log.cc:824] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:03.236419 10904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80: No bootstrap required, opened a new log
I20260809 02:10:03.238322 10904 raft_consensus.cc:348] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076d825dadfd4417a104a0167c76ef80" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33367 } }
I20260809 02:10:03.238495 10904 raft_consensus.cc:374] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:03.238520 10904 raft_consensus.cc:729] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 076d825dadfd4417a104a0167c76ef80, State: Initialized, Role: FOLLOWER
I20260809 02:10:03.238847 10904 consensus_queue.cc:260] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076d825dadfd4417a104a0167c76ef80" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33367 } }
I20260809 02:10:03.238988 10904 raft_consensus.cc:388] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:10:03.239035 10904 raft_consensus.cc:482] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:10:03.239090 10904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:03.239148 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 10835
I20260809 02:10:03.239269 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:10:03.239480 5158 external_mini_cluster.cc:1428] Setting key f8a2263f1efa4081862bb79bd4e78d99
I20260809 02:10:03.239959 10904 raft_consensus.cc:504] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076d825dadfd4417a104a0167c76ef80" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33367 } }
I20260809 02:10:03.240080 10904 leader_election.cc:302] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 076d825dadfd4417a104a0167c76ef80; no voters:
I20260809 02:10:03.240273 10904 leader_election.cc:288] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:10:03.240327 10910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:03.240480 10910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [term 1 LEADER]: Becoming Leader. State: Replica: 076d825dadfd4417a104a0167c76ef80, State: Running, Role: LEADER
I20260809 02:10:03.240576 10904 sys_catalog.cc:564] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:03.240582 10910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076d825dadfd4417a104a0167c76ef80" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33367 } }
I20260809 02:10:03.240995 10910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 076d825dadfd4417a104a0167c76ef80. Latest consensus state: current_term: 1 leader_uuid: "076d825dadfd4417a104a0167c76ef80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076d825dadfd4417a104a0167c76ef80" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33367 } } }
I20260809 02:10:03.241092 10910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:03.241003 10912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "076d825dadfd4417a104a0167c76ef80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076d825dadfd4417a104a0167c76ef80" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33367 } } }
I20260809 02:10:03.241240 10912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:03.241348 10918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:10:03.241945 10918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:10:03.244361 10918 catalog_manager.cc:1347] Generated new cluster ID: 094e4e2b4f7649f1888dc377122c3045
I20260809 02:10:03.244436 10918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:10:03.246413 10908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:03.240027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.5.9.190:33367)
0809 02:10:03.240272 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:03.240283 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:03.240324 (+ 41us) server_negotiation.cc:407] Connection header received
0809 02:10:03.240464 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:03.240473 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:03.240550 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:03.240996 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:03.241109 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:03.243009 (+ 1900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:03.243844 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:03.244468 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:03.244596 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:03.245656 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:03.245660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:03.245660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:03.245871 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:03.245915 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:03.245937 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:03.246004 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:03.246115 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 02:10:03.246195 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":90,"threads_started":1}
I20260809 02:10:03.247524 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:10:03.271327 10918 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:10:03.271921 10918 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:10:03.276633 10918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 076d825dadfd4417a104a0167c76ef80: Generated new TSK 0
I20260809 02:10:03.276882 10918 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:10:03.427894 10908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:03.290332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34710 (local address 127.5.9.190:33367)
0809 02:10:03.290415 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:03.290419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:03.290539 (+ 120us) server_negotiation.cc:407] Connection header received
0809 02:10:03.290654 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:03.290655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:03.290666 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:03.290756 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0809 02:10:03.290921 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:03.291477 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:03.292364 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:03.292614 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:03.292696 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:03.292919 (+ 223us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 02:10:03.427549 (+134630us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 02:10:03.427792 (+ 243us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34710: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 02:10:03.427995 10908 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34710: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 02:10:03.428004 10936 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:03.290255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:33367 (local address 127.0.0.1:34710)
0809 02:10:03.290452 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:03.290463 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:03.290554 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:03.290755 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:03.290756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:03.290759 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0809 02:10:03.290842 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:03.290844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:03.291596 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:03.291598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:03.292258 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:03.292261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:03.292696 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:03.292701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:03.292771 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:03.292783 (+ 12us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 02:10:03.427734 (+134951us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 02:10:03.427870 (+ 136us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.9.190:33367: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":122,"threads_started":1}
I20260809 02:10:03.428267 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 10835
2026-08-09T02:10:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (354 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (98 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (100 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:10:03 dist-test-slave-p3t5 krb5kdc[10949](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:10:03 dist-test-slave-p3t5 krb5kdc[10949](info): set up 2 sockets
Aug 09 02:10:03 dist-test-slave-p3t5 krb5kdc[10949](info): commencing operation
krb5kdc: starting...
W20260809 02:10:04.723259 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:10:04 dist-test-slave-p3t5 krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:10:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:10:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:10:04.875552 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:42013
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:39051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:42013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:04.963979 10963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:04.964282 10963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:04.964322 10963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:04.965582 10963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:04.965687 10963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:04.965713 10963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:04.965734 10963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:04.965759 10963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:10:04.967373 10963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:39051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:42013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:42013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:04.967682 10963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:04.968024 10963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:04.973658 10969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:04.973685 10971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:04.973649 10968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:04.974268 10963 server_base.cc:1034] running on GCE node
I20260809 02:10:04.974594 10963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:04.974998 10963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:04.976233 10963 hybrid_clock.cc:648] HybridClock initialized: now 1786241404976214 us; error 40 us; skew 500 ppm
Aug 09 02:10:04 dist-test-slave-p3t5 krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241404, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:04.979285 10963 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:10:04.979808 10963 webserver.cc:460] Webserver started at http://127.5.9.190:33809/ using document root <none> and password file <none>
I20260809 02:10:04.980080 10963 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:04.980156 10963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:04.980324 10963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:04.981487 10963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "b0527a71fd1d407ebbaf397596471cf0"
format_stamp: "Formatted at 2026-08-09 02:10:04 on dist-test-slave-p3t5"
server_key: "c57c487049bd5f97991818bf090488b3"
server_key_iv: "4ecfd2d173287ffafd6a939e34390d36"
server_key_version: "encryptionkey@0"
I20260809 02:10:04.981981 10963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "b0527a71fd1d407ebbaf397596471cf0"
format_stamp: "Formatted at 2026-08-09 02:10:04 on dist-test-slave-p3t5"
server_key: "c57c487049bd5f97991818bf090488b3"
server_key_iv: "4ecfd2d173287ffafd6a939e34390d36"
server_key_version: "encryptionkey@0"
I20260809 02:10:04.983513 10963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:04.984524 10978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:04.984812 10963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:04.984947 10963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "b0527a71fd1d407ebbaf397596471cf0"
format_stamp: "Formatted at 2026-08-09 02:10:04 on dist-test-slave-p3t5"
server_key: "c57c487049bd5f97991818bf090488b3"
server_key_iv: "4ecfd2d173287ffafd6a939e34390d36"
server_key_version: "encryptionkey@0"
I20260809 02:10:04.985059 10963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:05.009380 10963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:05.012914 10963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:05.013155 10963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:05.018441 10963 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:42013
I20260809 02:10:05.018523 11031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:42013 every 8 connection(s)
I20260809 02:10:05.018934 10963 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:10:05.019968 11032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:05.022680 11032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0: Bootstrap starting.
I20260809 02:10:05.022799 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 10963
I20260809 02:10:05.022897 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:10:05.023144 5158 external_mini_cluster.cc:1428] Setting key ef56625a639775bdb3323295232ea299
I20260809 02:10:05.023489 11032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:05.023929 11032 log.cc:824] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:05.024813 11032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0: No bootstrap required, opened a new log
I20260809 02:10:05.026917 11032 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0527a71fd1d407ebbaf397596471cf0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42013 } }
I20260809 02:10:05.027113 11032 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:05.027168 11032 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0527a71fd1d407ebbaf397596471cf0, State: Initialized, Role: FOLLOWER
I20260809 02:10:05.027316 11032 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0527a71fd1d407ebbaf397596471cf0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42013 } }
I20260809 02:10:05.027405 11032 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:10:05.027452 11032 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:10:05.027536 11032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:05.028190 11032 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0527a71fd1d407ebbaf397596471cf0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42013 } }
I20260809 02:10:05.028324 11032 leader_election.cc:302] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0527a71fd1d407ebbaf397596471cf0; no voters:
I20260809 02:10:05.028574 11032 leader_election.cc:288] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:10:05.028604 11037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:05.028800 11037 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [term 1 LEADER]: Becoming Leader. State: Replica: b0527a71fd1d407ebbaf397596471cf0, State: Running, Role: LEADER
I20260809 02:10:05.028873 11032 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:05.028915 11037 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0527a71fd1d407ebbaf397596471cf0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42013 } }
I20260809 02:10:05.029309 11039 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0527a71fd1d407ebbaf397596471cf0. Latest consensus state: current_term: 1 leader_uuid: "b0527a71fd1d407ebbaf397596471cf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0527a71fd1d407ebbaf397596471cf0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42013 } } }
I20260809 02:10:05.029405 11039 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:05.029717 11043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:10:05.030328 11043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:10:05.031272 11038 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0527a71fd1d407ebbaf397596471cf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0527a71fd1d407ebbaf397596471cf0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42013 } } }
I20260809 02:10:05.031374 11038 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:05.032490 11043 catalog_manager.cc:1347] Generated new cluster ID: 38cb50cb738844ad8c60984f022a021b
I20260809 02:10:05.032541 11043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:10:05.035871 11035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:05.023751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60842 (local address 127.5.9.190:42013)
0809 02:10:05.023970 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:05.023979 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:05.024008 (+ 29us) server_negotiation.cc:407] Connection header received
0809 02:10:05.024694 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:05.024703 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:05.024763 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:05.024932 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:05.025634 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.026586 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.027628 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.028088 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.028221 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:05.031722 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:05.031727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:05.031727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:05.031741 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:05.033929 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:05.034424 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:05.034428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:05.034428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:05.034546 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:05.034791 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:05.034792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:05.034792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:05.035127 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:05.035282 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:05.035427 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 02:10:05.035578 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":116,"threads_started":1}
I20260809 02:10:05.057080 11043 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:10:05.057904 11043 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:10:05.065857 11043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0527a71fd1d407ebbaf397596471cf0: Generated new TSK 0
I20260809 02:10:05.066098 11043 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:10:05.105561 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42013
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:39051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:05.194990 11063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:05.195235 11063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:05.195268 11063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:05.196507 11063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:05.196556 11063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:05.196585 11063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:10:05.198419 11063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:39051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:05.198707 11063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:05.198987 11063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:05.204620 11069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:05.204622 11071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:05.204620 11068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:05.205142 11063 server_base.cc:1034] running on GCE node
I20260809 02:10:05.205448 11063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:05.205750 11063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:05.206961 11063 hybrid_clock.cc:648] HybridClock initialized: now 1786241405206959 us; error 54 us; skew 500 ppm
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241405, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:05.210023 11063 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:10:05.210561 11063 webserver.cc:460] Webserver started at http://127.5.9.129:35807/ using document root <none> and password file <none>
I20260809 02:10:05.210824 11063 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:05.210875 11063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:05.211054 11063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:05.212329 11063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "ca1216a43a6341eba2d8bdf0409b593e"
format_stamp: "Formatted at 2026-08-09 02:10:05 on dist-test-slave-p3t5"
server_key: "a180d44d2119225819a60b9cff7aff89"
server_key_iv: "f8f769af5805d1214f88e079612e5438"
server_key_version: "encryptionkey@0"
I20260809 02:10:05.212726 11063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "ca1216a43a6341eba2d8bdf0409b593e"
format_stamp: "Formatted at 2026-08-09 02:10:05 on dist-test-slave-p3t5"
server_key: "a180d44d2119225819a60b9cff7aff89"
server_key_iv: "f8f769af5805d1214f88e079612e5438"
server_key_version: "encryptionkey@0"
I20260809 02:10:05.214382 11063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:10:05.215245 11078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:05.215598 11063 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:05.215713 11063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "ca1216a43a6341eba2d8bdf0409b593e"
format_stamp: "Formatted at 2026-08-09 02:10:05 on dist-test-slave-p3t5"
server_key: "a180d44d2119225819a60b9cff7aff89"
server_key_iv: "f8f769af5805d1214f88e079612e5438"
server_key_version: "encryptionkey@0"
I20260809 02:10:05.215834 11063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:05.254295 11063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:05.257048 11063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:05.257231 11063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:05.257542 11063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:05.257938 11063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:05.257977 11063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:05.258001 11063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:05.258014 11063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:05.264782 11063 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:45183
I20260809 02:10:05.264824 11195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:45183 every 8 connection(s)
I20260809 02:10:05.265260 11063 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241405, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:10:05.273204 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 11063
I20260809 02:10:05.273324 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:10:05.273558 5158 external_mini_cluster.cc:1428] Setting key 8baafe670b330872338c21b6d550d5a3
I20260809 02:10:05.277420 11035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:05.266256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:42475 (local address 127.5.9.190:42013)
0809 02:10:05.266323 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:05.266325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:05.266644 (+ 319us) server_negotiation.cc:407] Connection header received
0809 02:10:05.268148 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:05.268150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:05.268166 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:05.268241 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:05.268995 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.269517 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.270700 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.270930 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.271004 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:05.273588 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:05.273590 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:05.273590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:05.273600 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:05.275566 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:05.276141 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:05.276142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:05.276142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:05.276221 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:05.276672 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:05.276674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:05.276674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:05.276806 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:05.276867 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:05.277260 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 02:10:05.277344 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 02:10:05.277778 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:05.266068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:42013 (local address 127.5.9.129:42475)
0809 02:10:05.266372 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:05.266458 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:05.267970 (+ 1512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:05.268343 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:05.268346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:05.268656 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:05.268881 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:05.268886 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.269705 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.269707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:05.270603 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:05.270606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.271047 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.271048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:05.271166 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:05.271803 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:05.271816 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:05.273483 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:05.275711 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:05.275722 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:05.275727 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:05.276054 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:05.276357 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:05.276358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:05.276359 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:05.276549 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:05.276883 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:05.276891 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:05.277101 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 02:10:05.277235 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 02:10:05.277457 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":100,"threads_started":1}
I20260809 02:10:05.278317 11196 heartbeater.cc:347] Connected to a master server at 127.5.9.190:42013
I20260809 02:10:05.278460 11196 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:05.278695 11196 heartbeater.cc:511] Master 127.5.9.190:42013 requested a full tablet report, sending...
I20260809 02:10:05.279363 10995 ts_manager.cc:194] Registered new tserver with Master: ca1216a43a6341eba2d8bdf0409b593e (127.5.9.129:45183)
I20260809 02:10:05.280393 10995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:42475
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:10:05.305951 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42013
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:39051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:05.392854 11202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:05.393203 11202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:05.393236 11202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:05.394589 11202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:05.394645 11202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:05.394675 11202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:10:05.396337 11202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:39051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:05.396627 11202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:05.397007 11202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:05.403165 11208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:05.403362 11202 server_base.cc:1034] running on GCE node
W20260809 02:10:05.403151 11207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:05.403154 11210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:05.403713 11202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:05.404049 11202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:05.405272 11202 hybrid_clock.cc:648] HybridClock initialized: now 1786241405405266 us; error 42 us; skew 500 ppm
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241405, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:05.412389 11202 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:10:05.412981 11202 webserver.cc:460] Webserver started at http://127.5.9.130:37123/ using document root <none> and password file <none>
I20260809 02:10:05.413203 11202 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:05.413247 11202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:05.413360 11202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:05.414537 11202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "10fbca529e6c49f0ab612e3539caad14"
format_stamp: "Formatted at 2026-08-09 02:10:05 on dist-test-slave-p3t5"
server_key: "54526fa36d91dc688e6c99528f14e471"
server_key_iv: "393be48725bc631a29979f99a4b0959a"
server_key_version: "encryptionkey@0"
I20260809 02:10:05.414877 11202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "10fbca529e6c49f0ab612e3539caad14"
format_stamp: "Formatted at 2026-08-09 02:10:05 on dist-test-slave-p3t5"
server_key: "54526fa36d91dc688e6c99528f14e471"
server_key_iv: "393be48725bc631a29979f99a4b0959a"
server_key_version: "encryptionkey@0"
I20260809 02:10:05.416353 11202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:10:05.417191 11217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:05.417460 11202 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:10:05.417600 11202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "10fbca529e6c49f0ab612e3539caad14"
format_stamp: "Formatted at 2026-08-09 02:10:05 on dist-test-slave-p3t5"
server_key: "54526fa36d91dc688e6c99528f14e471"
server_key_iv: "393be48725bc631a29979f99a4b0959a"
server_key_version: "encryptionkey@0"
I20260809 02:10:05.417748 11202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:05.427337 11202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:05.430029 11202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:05.430219 11202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:05.430503 11202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:05.430871 11202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:05.430932 11202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:05.430979 11202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:05.431015 11202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:05.437772 11202 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:33151
I20260809 02:10:05.437820 11335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:33151 every 8 connection(s)
I20260809 02:10:05.438194 11202 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:10:05.442668 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 11202
I20260809 02:10:05.442878 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:10:05.443095 5158 external_mini_cluster.cc:1428] Setting key 7e78458947bbf642a446b378a53ece5b
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241405, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:10:05.449957 11035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:05.438992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:54125 (local address 127.5.9.190:42013)
0809 02:10:05.439053 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:05.439055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:05.439276 (+ 221us) server_negotiation.cc:407] Connection header received
0809 02:10:05.440290 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:05.440292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:05.440305 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:05.440376 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:05.441060 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.441577 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.442775 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.443141 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.443228 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:05.446295 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:05.446299 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:05.446299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:05.446313 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:05.448277 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:05.448775 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:05.448777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:05.448777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:05.448856 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:05.449155 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:05.449156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:05.449156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:05.449299 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:05.449373 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:05.449756 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 02:10:05.449855 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:10:05.450194 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:05.438937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:42013 (local address 127.5.9.130:54125)
0809 02:10:05.439121 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:05.439168 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:05.440158 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:05.440437 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:05.440441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:05.440738 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:05.440959 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:05.440964 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.441759 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.441760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:05.442648 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:05.442651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.443320 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.443321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:05.443493 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:05.444212 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:05.444225 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:05.446171 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:05.448379 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:05.448390 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:05.448394 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:05.448683 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:05.448929 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:05.448930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:05.448931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:05.449074 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:05.449377 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:05.449386 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:05.449571 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 02:10:05.449722 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 02:10:05.449917 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260809 02:10:05.450973 11336 heartbeater.cc:347] Connected to a master server at 127.5.9.190:42013
I20260809 02:10:05.451107 11336 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:05.451325 11336 heartbeater.cc:511] Master 127.5.9.190:42013 requested a full tablet report, sending...
I20260809 02:10:05.451848 10995 ts_manager.cc:194] Registered new tserver with Master: 10fbca529e6c49f0ab612e3539caad14 (127.5.9.130:33151)
I20260809 02:10:05.452423 10995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:54125
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:10:05.477581 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42013
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:39051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:05.569846 11343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:05.570112 11343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:05.570144 11343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:05.571610 11343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:05.571674 11343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:05.571707 11343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:10:05.573545 11343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:39051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:05.573884 11343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:05.574174 11343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:10:05.580291 11343 server_base.cc:1034] running on GCE node
W20260809 02:10:05.580202 11353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:05.580202 11350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:05.580211 11351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:05.580804 11343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:05.581045 11343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:05.582201 11343 hybrid_clock.cc:648] HybridClock initialized: now 1786241405582178 us; error 45 us; skew 500 ppm
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241405, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:05.585119 11343 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:10:05.585685 11343 webserver.cc:460] Webserver started at http://127.5.9.131:35749/ using document root <none> and password file <none>
I20260809 02:10:05.585973 11343 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:05.586024 11343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:05.586201 11343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:05.587364 11343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "a1e003d54c124a55a68e1c511b0d67ec"
format_stamp: "Formatted at 2026-08-09 02:10:05 on dist-test-slave-p3t5"
server_key: "239b439c014d434a2cdc9558131b8192"
server_key_iv: "63b58ea0415a665cd2d2e5cba6dc28dc"
server_key_version: "encryptionkey@0"
I20260809 02:10:05.587774 11343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "a1e003d54c124a55a68e1c511b0d67ec"
format_stamp: "Formatted at 2026-08-09 02:10:05 on dist-test-slave-p3t5"
server_key: "239b439c014d434a2cdc9558131b8192"
server_key_iv: "63b58ea0415a665cd2d2e5cba6dc28dc"
server_key_version: "encryptionkey@0"
I20260809 02:10:05.589393 11343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:10:05.590360 11361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:05.590617 11343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:05.590731 11343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "a1e003d54c124a55a68e1c511b0d67ec"
format_stamp: "Formatted at 2026-08-09 02:10:05 on dist-test-slave-p3t5"
server_key: "239b439c014d434a2cdc9558131b8192"
server_key_iv: "63b58ea0415a665cd2d2e5cba6dc28dc"
server_key_version: "encryptionkey@0"
I20260809 02:10:05.590824 11343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:05.613885 11343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:05.616657 11343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:05.616905 11343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:05.617223 11343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:05.617686 11343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:05.617767 11343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:05.617817 11343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:05.617877 11343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:05.625160 11343 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:46279
I20260809 02:10:05.625216 11479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:46279 every 8 connection(s)
I20260809 02:10:05.625603 11343 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241405, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:10:05.634812 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 11343
I20260809 02:10:05.634936 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:10:05.635185 5158 external_mini_cluster.cc:1428] Setting key 09b169b62b67696006f6bf723931abb8
I20260809 02:10:05.636382 11035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:05.626492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:38603 (local address 127.5.9.190:42013)
0809 02:10:05.626583 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:05.626586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:05.626751 (+ 165us) server_negotiation.cc:407] Connection header received
0809 02:10:05.627835 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:05.627836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:05.627851 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:05.627914 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:05.628610 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.629237 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.630420 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.630626 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.630717 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:05.633060 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:05.633061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:05.633061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:05.633070 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:05.634360 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:05.634938 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:05.634939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:05.634940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:05.635031 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:05.635473 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:05.635475 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:05.635475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:05.635675 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:05.635768 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:05.636178 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 02:10:05.636281 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:10:05.636559 11483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:05.626404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:42013 (local address 127.5.9.131:38603)
0809 02:10:05.626601 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:05.626642 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:05.627705 (+ 1063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:05.627997 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:05.628001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:05.628297 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:05.628506 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:05.628510 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.629336 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.629338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:05.630320 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:05.630323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.630729 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.630731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:05.630865 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:05.631420 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:05.631429 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:05.632938 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:05.634458 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:05.634466 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:05.634469 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:05.634816 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:05.635118 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:05.635119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:05.635119 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:05.635281 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:05.635749 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:05.635757 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:05.636008 (+ 251us) client_negotiation.cc:769] Sending connection context
0809 02:10:05.636132 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 02:10:05.636336 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":108,"threads_started":1}
I20260809 02:10:05.637074 11480 heartbeater.cc:347] Connected to a master server at 127.5.9.190:42013
I20260809 02:10:05.637190 11480 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:05.637385 11480 heartbeater.cc:511] Master 127.5.9.190:42013 requested a full tablet report, sending...
I20260809 02:10:05.637950 10995 ts_manager.cc:194] Registered new tserver with Master: a1e003d54c124a55a68e1c511b0d67ec (127.5.9.131:46279)
I20260809 02:10:05.638366 10995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:38603
I20260809 02:10:05.647497 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/test-admin.keytab.
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:05.667201 5158 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 02:10:05.668956 11488 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:05.668153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.5.9.129:45183)
0809 02:10:05.668351 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:05.668356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:05.668367 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:10:05.668396 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:05.668398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:05.668413 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:05.668496 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:05.668823 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38398: BlockingRecv error: recv got EOF from 127.0.0.1:38398 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":110,"threads_started":1}
W20260809 02:10:05.669122 5158 init.cc:244] Error while opening credential cache '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/krb5cc)
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:05.670214 5158 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:05.671372 5158 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.129@KRBTEST.COM
I20260809 02:10:05.680599 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:05.672156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38402 (local address 127.5.9.129:45183)
0809 02:10:05.672221 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:05.672223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:05.672322 (+ 99us) server_negotiation.cc:407] Connection header received
0809 02:10:05.672426 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:05.672427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:05.672439 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:05.672502 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:05.672956 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.673796 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.674585 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.674951 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.675039 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:05.677509 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:05.677520 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:05.677521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:05.677529 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:05.679208 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:05.679667 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:05.679671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:05.679671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:05.679747 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:05.679971 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:05.679972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:05.679972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:05.680138 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:05.680332 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:05.680379 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 02:10:05.680470 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 02:10:05.682696 11488 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:05.682166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38410 (local address 127.5.9.129:45183)
0809 02:10:05.682220 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:05.682222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:05.682231 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:10:05.682278 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:05.682279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:05.682289 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:05.682342 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:05.682643 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38410: BlockingRecv error: recv got EOF from 127.0.0.1:38410 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260809 02:10:05.682811 5158 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/krb5cc' of type FILE
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:05.683895 5158 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:05.684803 5158 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.129@KRBTEST.COM
I20260809 02:10:05.693234 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:05.685425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38422 (local address 127.5.9.129:45183)
0809 02:10:05.685490 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:05.685491 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:05.685591 (+ 100us) server_negotiation.cc:407] Connection header received
0809 02:10:05.685676 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:05.685677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:05.685693 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:05.685764 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:05.686164 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.686677 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.687489 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.687723 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.687793 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:05.690371 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:05.690373 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:05.690373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:05.690380 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:05.691730 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:05.692107 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:05.692109 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:05.692109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:05.692172 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:05.692640 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:05.692641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:05.692642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:05.692765 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:05.692862 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:05.693076 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 02:10:05.693151 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 02:10:05.695350 11488 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:05.694409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38426 (local address 127.5.9.129:45183)
0809 02:10:05.694470 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:05.694472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:05.694681 (+ 209us) server_negotiation.cc:407] Connection header received
0809 02:10:05.694737 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:05.694739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:05.694760 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:05.694846 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:05.695284 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38426: BlockingRecv error: recv got EOF from 127.0.0.1:38426 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 02:10:05.695391 5158 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241377393328-5158-0/krb5kdc/krb5cc' of type FILE
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:05.696550 5158 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:05.697499 5158 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[10949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.129@KRBTEST.COM
I20260809 02:10:05.706059 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:05.698253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38438 (local address 127.5.9.129:45183)
0809 02:10:05.698335 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:05.698337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:05.698526 (+ 189us) server_negotiation.cc:407] Connection header received
0809 02:10:05.698555 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:05.698556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:05.698567 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:05.698618 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:05.699035 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.699610 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.700509 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:05.700756 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:05.700833 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:05.703353 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:05.703355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:05.703355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:05.703363 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:05.704664 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:05.705055 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:05.705057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:05.705057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:05.705125 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:05.705360 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:05.705361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:05.705361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:05.705485 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:05.705577 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:05.705777 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 02:10:05.705852 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 02:10:05.706820 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 11063
I20260809 02:10:05.714102 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 11202
I20260809 02:10:05.722260 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 11343
I20260809 02:10:05.728461 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 10963
2026-08-09T02:10:05Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2104 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[11504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[11504](info): set up 2 sockets
Aug 09 02:10:05 dist-test-slave-p3t5 krb5kdc[11504](info): commencing operation
krb5kdc: starting...
W20260809 02:10:06.830506 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:10:06 dist-test-slave-p3t5 krb5kdc[11504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:10:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:10:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.190.keytab.
Entry for principal oryx/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.190.keytab.
I20260809 02:10:06.976610 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:39773
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.190.keytab
--principal=oryx/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:39773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:07.066811 11520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:07.067113 11520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:07.067157 11520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:07.068516 11520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:07.068610 11520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:07.068636 11520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:07.068658 11520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:07.068687 11520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:10:07.070528 11520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:39773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.190.keytab
--principal=oryx/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:39773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:07.070868 11520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:07.071165 11520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:07.076674 11526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:07.076658 11528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:07.076751 11525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:07.077168 11520 server_base.cc:1034] running on GCE node
I20260809 02:10:07.077497 11520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:07.077899 11520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:07.079063 11520 hybrid_clock.cc:648] HybridClock initialized: now 1786241407079046 us; error 30 us; skew 500 ppm
Aug 09 02:10:07 dist-test-slave-p3t5 krb5kdc[11504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241407, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:07.082231 11520 init.cc:377] Logged in from keytab as oryx/127.5.9.190@KRBTEST.COM (short username oryx)
I20260809 02:10:07.082800 11520 webserver.cc:460] Webserver started at http://127.5.9.190:34789/ using document root <none> and password file <none>
I20260809 02:10:07.083014 11520 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:07.083060 11520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:07.083161 11520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:07.084345 11520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "86d06056e8354223aff7fe1f4a8cc361"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "e42c0139a7f074885ed97b114ae592fe"
server_key_iv: "d8f6a5c7d48a257aec1d6f741be8ccff"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.084702 11520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "86d06056e8354223aff7fe1f4a8cc361"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "e42c0139a7f074885ed97b114ae592fe"
server_key_iv: "d8f6a5c7d48a257aec1d6f741be8ccff"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.086310 11520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:10:07.087265 11536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:07.087512 11520 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:10:07.087616 11520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "86d06056e8354223aff7fe1f4a8cc361"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "e42c0139a7f074885ed97b114ae592fe"
server_key_iv: "d8f6a5c7d48a257aec1d6f741be8ccff"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.087688 11520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:07.102751 11520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:07.105381 11520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:07.105517 11520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:07.109977 11520 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:39773
I20260809 02:10:07.110044 11593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:39773 every 8 connection(s)
I20260809 02:10:07.110435 11520 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:10:07.111599 11594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:07.113379 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 11520
I20260809 02:10:07.113511 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:10:07.113763 5158 external_mini_cluster.cc:1428] Setting key ce062b138dda5ea274f3513b60cfb8d4
I20260809 02:10:07.114681 11594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361: Bootstrap starting.
I20260809 02:10:07.115386 11594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:07.115878 11594 log.cc:824] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:07.116791 11594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361: No bootstrap required, opened a new log
I20260809 02:10:07.119292 11594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } }
I20260809 02:10:07.119529 11594 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:07.119616 11594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86d06056e8354223aff7fe1f4a8cc361, State: Initialized, Role: FOLLOWER
I20260809 02:10:07.119845 11594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } }
I20260809 02:10:07.119971 11594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:10:07.120010 11594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:10:07.120065 11594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:07.120980 11594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } }
I20260809 02:10:07.121119 11594 leader_election.cc:302] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86d06056e8354223aff7fe1f4a8cc361; no voters:
I20260809 02:10:07.121309 11594 leader_election.cc:288] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:10:07.121349 11599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:07.121537 11599 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [term 1 LEADER]: Becoming Leader. State: Replica: 86d06056e8354223aff7fe1f4a8cc361, State: Running, Role: LEADER
I20260809 02:10:07.121649 11594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 02:10:07 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM
I20260809 02:10:07.121721 11599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } }
I20260809 02:10:07.122073 11599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86d06056e8354223aff7fe1f4a8cc361. Latest consensus state: current_term: 1 leader_uuid: "86d06056e8354223aff7fe1f4a8cc361" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } }
I20260809 02:10:07.122251 11599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:07.122833 11608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:10:07.123003 11601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86d06056e8354223aff7fe1f4a8cc361" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } }
I20260809 02:10:07.123507 11608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:10:07.123534 11601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:07.125602 11608 catalog_manager.cc:1347] Generated new cluster ID: 8c80a661add343e39c95dbfcedfb468e
I20260809 02:10:07.125697 11608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:10:07.126605 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.114383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49780 (local address 127.5.9.190:39773)
0809 02:10:07.114578 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:07.114588 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:07.114617 (+ 29us) server_negotiation.cc:407] Connection header received
0809 02:10:07.115182 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:07.115190 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:07.115248 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:07.115378 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:07.116429 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.117438 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.118582 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.119032 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.119119 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.122331 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:07.122336 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:07.122336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:07.122350 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:07.124666 (+ 2316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.125139 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.125142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.125143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.125258 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.125517 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.125518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.125518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.125897 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:07.126048 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:07.126200 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 02:10:07.126368 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260809 02:10:07.144052 11608 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:10:07.144853 11608 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:10:07.148505 11608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361: Generated new TSK 0
I20260809 02:10:07.148761 11608 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.129.keytab.
Entry for principal oryx/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.129.keytab.
I20260809 02:10:07.185423 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:39773
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.129.keytab
--principal=oryx/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:07.282711 11622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:07.282972 11622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:07.283000 11622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:07.284291 11622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:07.284340 11622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:07.284369 11622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:10:07.286089 11622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.129.keytab
--principal=oryx/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:39773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:07.286402 11622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:07.286670 11622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:07.292281 11627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:07.292282 11628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:07.292281 11631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:07.292501 11622 server_base.cc:1034] running on GCE node
I20260809 02:10:07.292719 11622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:07.292991 11622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:07.294139 11622 hybrid_clock.cc:648] HybridClock initialized: now 1786241407294128 us; error 27 us; skew 500 ppm
Aug 09 02:10:07 dist-test-slave-p3t5 krb5kdc[11504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241407, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:07.296919 11622 init.cc:377] Logged in from keytab as oryx/127.5.9.129@KRBTEST.COM (short username oryx)
I20260809 02:10:07.297432 11622 webserver.cc:460] Webserver started at http://127.5.9.129:46347/ using document root <none> and password file <none>
I20260809 02:10:07.297752 11622 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:07.297833 11622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:07.298003 11622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:07.299245 11622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "3955f153a3fb49cfb79a720b1ae457b5"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "e605fa0e34927605b3e0be77dd160cef"
server_key_iv: "11753db9e7caa024ebbd1eb74ab49f3b"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.299727 11622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "3955f153a3fb49cfb79a720b1ae457b5"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "e605fa0e34927605b3e0be77dd160cef"
server_key_iv: "11753db9e7caa024ebbd1eb74ab49f3b"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.301250 11622 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:10:07.302345 11638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:07.302579 11622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:10:07.302697 11622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "3955f153a3fb49cfb79a720b1ae457b5"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "e605fa0e34927605b3e0be77dd160cef"
server_key_iv: "11753db9e7caa024ebbd1eb74ab49f3b"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.302829 11622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:07.317946 11622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:07.320713 11622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:07.320907 11622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:07.321179 11622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:07.321522 11622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:07.321579 11622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:07.321648 11622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:07.321671 11622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:07.328478 11622 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:45209
I20260809 02:10:07.328523 11761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:45209 every 8 connection(s)
I20260809 02:10:07.328953 11622 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:10:07.332826 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 11622
I20260809 02:10:07.332928 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:10:07.333164 5158 external_mini_cluster.cc:1428] Setting key cc2fd0241eb85c2f99ca945df73c26c5
Aug 09 02:10:07 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241407, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM
I20260809 02:10:07.339969 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.330031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:53655 (local address 127.5.9.190:39773)
0809 02:10:07.330126 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:07.330128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:07.330299 (+ 171us) server_negotiation.cc:407] Connection header received
0809 02:10:07.330806 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:07.330807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:07.330817 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:07.330874 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:07.331873 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.332395 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.333779 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.334099 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.334179 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.336661 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:07.336663 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:07.336663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:07.336672 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:07.338146 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.338704 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.338705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.338706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.338771 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.339066 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.339067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.339067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.339202 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:07.339280 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:07.339682 (+ 402us) server_negotiation.cc:299] Negotiation successful
0809 02:10:07.339806 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:10:07.340134 11764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.329969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.129:53655)
0809 02:10:07.330159 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:07.330200 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:07.330715 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:07.330934 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:07.330937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:07.331576 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:07.331784 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.331789 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.332506 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.332507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.333692 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.333697 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.334197 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.334198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.334311 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.334914 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:07.334930 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:07.336543 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:07.338241 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.338250 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.338253 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.338606 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.338853 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.338854 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.338854 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.338990 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.339283 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:07.339292 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:07.339450 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 02:10:07.339621 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 02:10:07.339838 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
I20260809 02:10:07.340983 11762 heartbeater.cc:347] Connected to a master server at 127.5.9.190:39773
I20260809 02:10:07.341106 11762 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:07.341331 11762 heartbeater.cc:511] Master 127.5.9.190:39773 requested a full tablet report, sending...
I20260809 02:10:07.341995 11553 ts_manager.cc:194] Registered new tserver with Master: 3955f153a3fb49cfb79a720b1ae457b5 (127.5.9.129:45209)
I20260809 02:10:07.342633 11553 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:53655
WARNING: no policy specified for oryx/127.5.9.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.130.keytab.
Entry for principal oryx/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.130.keytab.
I20260809 02:10:07.365130 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:39773
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.130.keytab
--principal=oryx/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:07.455509 11767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:07.455766 11767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:07.455796 11767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:07.457223 11767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:07.457285 11767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:07.457316 11767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:10:07.459108 11767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.130.keytab
--principal=oryx/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:39773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:07.459419 11767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:07.459697 11767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:07.465308 11776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:07.465308 11772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:07.465375 11774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:07.465545 11767 server_base.cc:1034] running on GCE node
I20260809 02:10:07.465853 11767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:07.466147 11767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:07.467371 11767 hybrid_clock.cc:648] HybridClock initialized: now 1786241407467314 us; error 56 us; skew 500 ppm
Aug 09 02:10:07 dist-test-slave-p3t5 krb5kdc[11504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241407, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:07.470329 11767 init.cc:377] Logged in from keytab as oryx/127.5.9.130@KRBTEST.COM (short username oryx)
I20260809 02:10:07.470880 11767 webserver.cc:460] Webserver started at http://127.5.9.130:42097/ using document root <none> and password file <none>
I20260809 02:10:07.471184 11767 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:07.471290 11767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:07.471472 11767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:07.472779 11767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "b2976b42ac924f29bc197e23f317edb4"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "ff7ff0b01da4b1c02396a825f39d7a6f"
server_key_iv: "468bc77b22d63d70193b6d58a0e90d8e"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.473206 11767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "b2976b42ac924f29bc197e23f317edb4"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "ff7ff0b01da4b1c02396a825f39d7a6f"
server_key_iv: "468bc77b22d63d70193b6d58a0e90d8e"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.475133 11767 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 02:10:07.476145 11784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:07.476399 11767 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:07.476524 11767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "b2976b42ac924f29bc197e23f317edb4"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "ff7ff0b01da4b1c02396a825f39d7a6f"
server_key_iv: "468bc77b22d63d70193b6d58a0e90d8e"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.476626 11767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:07.488166 11767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:07.490720 11767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:07.490893 11767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:07.491180 11767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:07.491596 11767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:07.491654 11767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:07.491698 11767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:07.491726 11767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:07.498059 11767 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:46733
I20260809 02:10:07.498150 11902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:46733 every 8 connection(s)
I20260809 02:10:07.498526 11767 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:10:07.502345 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 11767
I20260809 02:10:07.502478 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:10:07.502665 5158 external_mini_cluster.cc:1428] Setting key d555da9a378e9bea09bc820fd9b75045
Aug 09 02:10:07 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241407, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM
I20260809 02:10:07.509758 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.499392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:52137 (local address 127.5.9.190:39773)
0809 02:10:07.499444 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:07.499446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:07.499692 (+ 246us) server_negotiation.cc:407] Connection header received
0809 02:10:07.500197 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:07.500198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:07.500210 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:07.500268 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:07.501288 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.501894 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.503264 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.503595 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.503674 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.506582 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:07.506583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:07.506583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:07.506594 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:07.508194 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.508663 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.508664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.508664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.508730 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.509014 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.509015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.509016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.509146 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:07.509221 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:07.509529 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 02:10:07.509648 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:10:07.510017 11905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.499360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.130:52137)
0809 02:10:07.499535 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:07.499591 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:07.500090 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:07.500331 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:07.500335 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:07.500979 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:07.501188 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.501193 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.501995 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.501996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.503154 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.503157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.503703 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.503704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.503872 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.504481 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:07.504493 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:07.506452 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:07.508305 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.508315 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.508318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.508576 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.508812 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.508812 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.508813 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.508942 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.509224 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:07.509232 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:07.509389 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 02:10:07.509504 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 02:10:07.509751 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
I20260809 02:10:07.510494 11903 heartbeater.cc:347] Connected to a master server at 127.5.9.190:39773
I20260809 02:10:07.510599 11903 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:07.510819 11903 heartbeater.cc:511] Master 127.5.9.190:39773 requested a full tablet report, sending...
I20260809 02:10:07.511240 11553 ts_manager.cc:194] Registered new tserver with Master: b2976b42ac924f29bc197e23f317edb4 (127.5.9.130:46733)
I20260809 02:10:07.511660 11553 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:52137
WARNING: no policy specified for oryx/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.131.keytab.
Entry for principal oryx/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.131.keytab.
I20260809 02:10:07.533753 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:39773
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.131.keytab
--principal=oryx/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:07.621593 11908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:07.621841 11908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:07.621869 11908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:07.623260 11908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:07.623314 11908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:07.623373 11908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:10:07.625103 11908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.131.keytab
--principal=oryx/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:39773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:07.625334 11908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:07.625557 11908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:07.631064 11916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:07.631064 11913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:07.631276 11908 server_base.cc:1034] running on GCE node
W20260809 02:10:07.631079 11914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:07.631553 11908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:07.631780 11908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:07.632936 11908 hybrid_clock.cc:648] HybridClock initialized: now 1786241407632919 us; error 31 us; skew 500 ppm
Aug 09 02:10:07 dist-test-slave-p3t5 krb5kdc[11504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241407, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:07.635807 11908 init.cc:377] Logged in from keytab as oryx/127.5.9.131@KRBTEST.COM (short username oryx)
I20260809 02:10:07.636317 11908 webserver.cc:460] Webserver started at http://127.5.9.131:42315/ using document root <none> and password file <none>
I20260809 02:10:07.636590 11908 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:07.636665 11908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:07.636827 11908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:07.638038 11908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "d811603a56444215834731cbff681ca1"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "c518afda707c5c7ced1d34218ebbe91f"
server_key_iv: "182c6aa6e8d9d2c3af4fec3a9bb075b2"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.638420 11908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "d811603a56444215834731cbff681ca1"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "c518afda707c5c7ced1d34218ebbe91f"
server_key_iv: "182c6aa6e8d9d2c3af4fec3a9bb075b2"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.639993 11908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 02:10:07.640833 11924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:07.641024 11908 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:07.641139 11908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "d811603a56444215834731cbff681ca1"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "c518afda707c5c7ced1d34218ebbe91f"
server_key_iv: "182c6aa6e8d9d2c3af4fec3a9bb075b2"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.641223 11908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:07.651122 11908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:07.653828 11908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:07.654014 11908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:07.654281 11908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:07.654629 11908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:07.654687 11908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:07.654731 11908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:07.654762 11908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:07.661221 11908 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:35771
I20260809 02:10:07.661253 12043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:35771 every 8 connection(s)
I20260809 02:10:07.661791 11908 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
Aug 09 02:10:07 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241407, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM
I20260809 02:10:07.670357 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 11908
I20260809 02:10:07.670487 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:10:07.670701 5158 external_mini_cluster.cc:1428] Setting key ef3285f05a567656c7371e0ba491c335
I20260809 02:10:07.672574 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.662610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:56579 (local address 127.5.9.190:39773)
0809 02:10:07.662679 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:07.662681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:07.662888 (+ 207us) server_negotiation.cc:407] Connection header received
0809 02:10:07.663481 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:07.663482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:07.663495 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:07.663540 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:07.664671 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.665199 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.666342 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.666565 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.666658 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.669009 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:07.669010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:07.669011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:07.669020 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:07.670936 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.671511 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.671512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.671513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.671591 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.671899 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.671900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.671900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.672048 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:07.672110 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:07.672407 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 02:10:07.672485 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:10:07.672736 12047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.662590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.131:56579)
0809 02:10:07.662751 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:07.662797 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:07.663345 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:07.663622 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:07.663627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:07.664305 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:07.664530 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.664536 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.665310 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.665311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.666246 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.666249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.666668 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.666669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.666817 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.667363 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:07.667371 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:07.668896 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:07.671022 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.671030 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.671033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.671432 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.671661 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.671662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.671663 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.671818 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.672116 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:07.672121 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:07.672253 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 02:10:07.672343 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 02:10:07.672504 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
I20260809 02:10:07.673262 12045 heartbeater.cc:347] Connected to a master server at 127.5.9.190:39773
I20260809 02:10:07.673353 12045 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:07.673508 12045 heartbeater.cc:511] Master 127.5.9.190:39773 requested a full tablet report, sending...
I20260809 02:10:07.673954 11553 ts_manager.cc:194] Registered new tserver with Master: d811603a56444215834731cbff681ca1 (127.5.9.131:35771)
I20260809 02:10:07.674316 11553 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:56579
I20260809 02:10:07.683007 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.9.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.9.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.9.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.189.keytab.
Entry for principal oryx/127.5.9.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.189.keytab.
I20260809 02:10:07.714365 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.189:40825
--webserver_interface=127.5.9.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.189.keytab
--principal=oryx/127.5.9.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:39773,127.5.9.189:40825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:07.801717 12050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:07.801971 12050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:07.801997 12050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:07.803463 12050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:07.803550 12050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:07.803567 12050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:07.803581 12050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:07.803594 12050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:10:07.805305 12050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:39773,127.5.9.189:40825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.189.keytab
--principal=oryx/127.5.9.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.189:40825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.9.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
W20260809 02:10:07.805433 12050 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.5.9.190:39773,127.5.9.189:40825'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 02:10:07.805651 12050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:07.805892 12050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:07.810984 12056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:07.810979 12058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:07.811025 12055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:07.811288 12050 server_base.cc:1034] running on GCE node
I20260809 02:10:07.811570 12050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:07.811887 12050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:07.813094 12050 hybrid_clock.cc:648] HybridClock initialized: now 1786241407813085 us; error 39 us; skew 500 ppm
Aug 09 02:10:07 dist-test-slave-p3t5 krb5kdc[11504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241407, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:07.815855 12050 init.cc:377] Logged in from keytab as oryx/127.5.9.189@KRBTEST.COM (short username oryx)
I20260809 02:10:07.816380 12050 webserver.cc:460] Webserver started at http://127.5.9.189:33895/ using document root <none> and password file <none>
I20260809 02:10:07.816633 12050 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:07.816706 12050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:07.816870 12050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:07.818145 12050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/data/instance:
uuid: "5fc6594b92bd463ca3ad7d723123c664"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "c39dce0bf3e3e836b22d60a95245a6d9"
server_key_iv: "1ac75bd8bce15112b28289eeba8769d1"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.818584 12050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/wal/instance:
uuid: "5fc6594b92bd463ca3ad7d723123c664"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "c39dce0bf3e3e836b22d60a95245a6d9"
server_key_iv: "1ac75bd8bce15112b28289eeba8769d1"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.820223 12050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:10:07.821175 12066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:07.821584 12050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:07.821784 12050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/wal
uuid: "5fc6594b92bd463ca3ad7d723123c664"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "c39dce0bf3e3e836b22d60a95245a6d9"
server_key_iv: "1ac75bd8bce15112b28289eeba8769d1"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.821862 12050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:07.872354 12050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:07.874861 12050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:07.874994 12050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:07.879325 12050 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.189:40825
I20260809 02:10:07.879374 12123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.189:40825 every 8 connection(s)
I20260809 02:10:07.879876 12050 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/data/info.pb
I20260809 02:10:07.881027 12124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:07.881366 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 12050
I20260809 02:10:07.883877 12124 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:10:07 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.9.189@KRBTEST.COM
Aug 09 02:10:07 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241407, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.189@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM
I20260809 02:10:07.893807 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.884735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49790 (local address 127.5.9.190:39773)
0809 02:10:07.884814 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:07.884816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:07.885003 (+ 187us) server_negotiation.cc:407] Connection header received
0809 02:10:07.885058 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:07.885059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:07.885072 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:07.885130 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:07.885746 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.886309 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.887312 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.887511 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.887621 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.890433 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:07.890435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:07.890435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:07.890445 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:07.892260 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.892671 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.892672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.892673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.892753 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.893020 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.893021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.893021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.893209 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:07.893312 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:07.893544 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 02:10:07.893671 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:10:07.894003 12128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.884670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.0.0.1:49790)
0809 02:10:07.884880 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:07.884929 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:07.884991 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:07.885161 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:07.885163 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:07.885530 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:07.885642 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.885645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.886427 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.886428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.887222 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.887225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.887632 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.887633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.887718 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.888226 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:07.888235 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:07.890329 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:07.892340 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.892347 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.892349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.892570 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.892829 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.892830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.892830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.892941 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.893299 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:07.893306 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:07.893404 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 02:10:07.893508 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 02:10:07.893798 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":94,"thread_start_us":107,"threads_started":1}
I20260809 02:10:07.894004 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.882168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58914 (local address 127.5.9.189:40825)
0809 02:10:07.882396 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:07.882405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:07.882434 (+ 29us) server_negotiation.cc:407] Connection header received
0809 02:10:07.882967 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:07.882977 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:07.883055 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:07.883242 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:07.883899 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.885253 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.886184 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.886703 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.886804 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.889786 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:07.889794 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:07.889795 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:07.889803 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:07.892174 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.892605 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.892608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.892609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.892707 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.892977 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.892978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.892978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.893224 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:07.893404 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:07.893492 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 02:10:07.893733 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":167,"thread_start_us":132,"threads_started":1}
I20260809 02:10:07.894905 12124 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:10:07 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241407, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.189@KRBTEST.COM for oryx/127.5.9.189@KRBTEST.COM
I20260809 02:10:07.902207 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.895313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58930 (local address 127.5.9.189:40825)
0809 02:10:07.895370 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:07.895371 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:07.895380 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:10:07.895397 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:07.895397 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:07.895406 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:07.895461 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:07.895753 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.896239 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.896927 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.897152 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.897240 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.899359 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:07.899361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:07.899361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:07.899368 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:07.900890 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.901296 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.901297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.901297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.901359 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.901570 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.901571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.901571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.901749 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:07.901827 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:07.901967 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 02:10:07.902057 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:10:07.902272 12128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.895202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:40825 (local address 127.0.0.1:58930)
0809 02:10:07.895256 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:07.895267 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:07.895314 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:07.895464 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:07.895465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:07.895615 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:07.895669 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.895670 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.896327 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.896328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.896826 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.896827 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.897239 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.897240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.897312 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.897851 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:07.897857 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:07.899222 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:07.901012 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.901013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.901013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.901190 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.901434 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.901435 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.901435 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.901500 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.901818 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:07.901818 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:07.901859 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 02:10:07.901946 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 02:10:07.902128 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":89}
I20260809 02:10:07.903823 12124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: Bootstrap starting.
I20260809 02:10:07.904773 12124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:07.905087 12124 log.cc:824] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:07.906497 12124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: No bootstrap required, opened a new log
I20260809 02:10:07.908254 12124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } }
I20260809 02:10:07.908442 12124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:07.908497 12124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc6594b92bd463ca3ad7d723123c664, State: Initialized, Role: FOLLOWER
I20260809 02:10:07.908660 12124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } }
I20260809 02:10:07.909035 12124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:07.909049 12130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } } }
I20260809 02:10:07.909215 12130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 02:10:07.911085 12141 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:10:07.911156 12141 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:10:07.913316 12050 master_runner.cc:428] Detected that this master 5fc6594b92bd463ca3ad7d723123c664 is joining an existing cluster
I20260809 02:10:07.913426 12050 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.9.189:40825
I20260809 02:10:07.922271 12148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.914719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:40825 (local address 127.0.0.1:58938)
0809 02:10:07.915061 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:07.915072 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:07.915126 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:07.915270 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:07.915271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:07.915518 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:07.915580 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.915582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.916477 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.916477 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.917296 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.917298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.917833 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.917834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.917902 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.918297 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:07.918303 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:07.918724 (+ 421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:07.920817 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.920818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.920818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.921098 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.921309 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.921310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.921310 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.921383 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.921722 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:07.921723 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:07.921769 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 02:10:07.921849 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 02:10:07.922031 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":104,"threads_started":1}
I20260809 02:10:07.922292 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.914605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58938 (local address 127.5.9.189:40825)
0809 02:10:07.914653 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:07.914655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:07.915139 (+ 484us) server_negotiation.cc:407] Connection header received
0809 02:10:07.915182 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:07.915183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:07.915193 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:07.915250 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:07.915673 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.916389 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.917380 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.917740 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.917834 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.918812 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:07.918813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:07.918814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:07.918820 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:07.920713 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.921163 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.921164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.921165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.921231 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.921465 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.921466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.921466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.921592 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:07.921698 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:07.921907 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 02:10:07.922106 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":25}
I20260809 02:10:07.922442 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.914655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49796 (local address 127.5.9.190:39773)
0809 02:10:07.914694 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:07.914696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:07.914856 (+ 160us) server_negotiation.cc:407] Connection header received
0809 02:10:07.914913 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:07.914915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:07.914927 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:07.914982 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:07.915457 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.916031 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.916914 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.917144 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.917239 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.918725 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:07.918726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:07.918727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:07.918735 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:07.920766 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.921362 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.921363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.921363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.921433 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.921703 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.921704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.921704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.921886 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:07.921978 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:07.922235 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 02:10:07.922334 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 02:10:07.923211 12147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.914586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.0.0.1:49796)
0809 02:10:07.914763 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:07.914777 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:07.914852 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:07.914984 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:07.914985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:07.915258 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:07.915363 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.915365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.916130 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.916131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.916809 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.916811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.917265 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.917266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.917358 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.918067 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:07.918071 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:07.918608 (+ 537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:07.920871 (+ 2263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.920872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.920872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.921265 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.921508 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.921508 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.921509 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.921572 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.922027 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:07.922028 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:07.922085 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 02:10:07.922960 (+ 875us) client_negotiation.cc:240] Negotiation successful
0809 02:10:07.923110 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":112,"threads_started":1}
Aug 09 02:10:07 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241407, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.190@KRBTEST.COM for oryx/127.5.9.189@KRBTEST.COM
I20260809 02:10:07.935045 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.925906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58954 (local address 127.5.9.189:40825)
0809 02:10:07.925950 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:07.925952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:07.926180 (+ 228us) server_negotiation.cc:407] Connection header received
0809 02:10:07.926247 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:07.926248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:07.926258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:07.926320 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:07.926765 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.927563 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.928750 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.929069 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.929145 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.931778 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:07.931780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:07.931780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:07.931788 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:07.933309 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.933932 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.933935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.933936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.934075 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.934422 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.934423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.934424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.934605 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:07.934676 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:07.934900 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 02:10:07.934970 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:10:07.935163 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.925873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:40825 (local address 127.0.0.1:58954)
0809 02:10:07.926048 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:07.926105 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:07.926182 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:07.926329 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:07.926333 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:07.926628 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:07.926685 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.926687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.927734 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.927736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.928651 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.928656 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.929184 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.929185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.929289 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.929958 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:07.929963 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:07.931647 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:07.933414 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.933426 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.933429 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.933775 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.934202 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.934203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.934203 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.934289 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.934691 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:07.934701 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:07.934775 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 02:10:07.934880 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 02:10:07.935050 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":117,"threads_started":1}
I20260809 02:10:07.935921 11553 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.9.189:40825: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664"
member_type: NON_VOTER
last_known_addr {
host: "127.5.9.189"
port: 40825
}
attrs {
promote: true
}
}
dest_uuid: "86d06056e8354223aff7fe1f4a8cc361"
cas_config_opid_index: -1
I20260809 02:10:07.936340 11553 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: NON_VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: true } }
I20260809 02:10:07.936926 12151 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5fc6594b92bd463ca3ad7d723123c664 (127.5.9.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: NON_VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: true } } }
I20260809 02:10:07.937255 12094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:07.938225 12151 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.9.189:40825
I20260809 02:10:07.938265 12094 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 1 FOLLOWER]: Refusing update from remote peer 86d06056e8354223aff7fe1f4a8cc361: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 02:10:07.938536 12050 master.cc:537] Master@127.5.9.189:40825 shutting down...
I20260809 02:10:07.938788 12151 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: NON_VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:10:07.939620 11538 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260809 02:10:07.939689 12150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "86d06056e8354223aff7fe1f4a8cc361" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: NON_VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: true } } }
I20260809 02:10:07.939785 12150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: This master's current role is: LEADER
W20260809 02:10:07.940186 11538 consensus_peers.cc:487] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 -> Peer 5fc6594b92bd463ca3ad7d723123c664 (127.5.9.189:40825): Couldn't send request to peer 5fc6594b92bd463ca3ad7d723123c664. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 02:10:07.941681 12050 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:07.941781 12050 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:07.941855 12050 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: stopping tablet replica
I20260809 02:10:07.955108 12050 master.cc:552] Master@127.5.9.189:40825 shutdown complete.
I20260809 02:10:07.955175 12050 master_runner.cc:305] Clearing existing system tablet
I20260809 02:10:07.955366 12050 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:07.955427 12050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:07.956203 12050 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.002s
I20260809 02:10:07.956795 12156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:07.956938 12050 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:07.957003 12050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 02:10:07.957036 12050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/wal
uuid: "5fc6594b92bd463ca3ad7d723123c664"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "c39dce0bf3e3e836b22d60a95245a6d9"
server_key_iv: "1ac75bd8bce15112b28289eeba8769d1"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.957957 12050 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:07.959823 12050 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 02:10:07.960032 12050 log.cc:1196] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 02:10:07.960319 12050 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: Deleting consensus metadata
I20260809 02:10:07.960489 12050 master_runner.cc:315] Copying system tablet from 127.5.9.190:39773
I20260809 02:10:07.961742 12050 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: tablet copy: Beginning tablet copy session from remote peer at address 127.5.9.190:39773
I20260809 02:10:07.969579 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.962296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49806 (local address 127.5.9.190:39773)
0809 02:10:07.962347 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:07.962349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:07.962389 (+ 40us) server_negotiation.cc:407] Connection header received
0809 02:10:07.962442 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:07.962443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:07.962454 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:07.962503 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:07.962933 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.963458 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.964582 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.964810 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.964918 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.966158 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:07.966161 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:07.966162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:07.966174 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:07.968219 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.968701 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.968702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.968703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.968779 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:07.969019 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:07.969020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:07.969021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:07.969158 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:07.969225 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:07.969423 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 02:10:07.969504 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:10:07.969695 12162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:07.962168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.0.0.1:49806)
0809 02:10:07.962320 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:07.962331 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:07.962376 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:07.962530 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:07.962531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:07.962772 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:07.962857 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.962859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.963544 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.963546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.964471 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:07.964475 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:07.964897 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:07.964898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:07.964977 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:07.965515 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:07.965521 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:07.965977 (+ 456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:07.968324 (+ 2347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.968325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.968326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.968598 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.968857 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:07.968857 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:07.968858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:07.968930 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:07.969238 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:07.969239 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:07.969310 (+ 71us) client_negotiation.cc:769] Sending connection context
0809 02:10:07.969386 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:10:07.969507 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260809 02:10:07.970225 11575 tablet_copy_service.cc:140] P 86d06056e8354223aff7fe1f4a8cc361: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5fc6594b92bd463ca3ad7d723123c664 ({username='oryx', principal='oryx/127.5.9.189@KRBTEST.COM'} at 127.0.0.1:49806)
I20260809 02:10:07.970379 11575 tablet_copy_service.cc:161] P 86d06056e8354223aff7fe1f4a8cc361: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5fc6594b92bd463ca3ad7d723123c664 at {username='oryx', principal='oryx/127.5.9.189@KRBTEST.COM'} at 127.0.0.1:49806: session id = 5fc6594b92bd463ca3ad7d723123c664-00000000000000000000000000000000
I20260809 02:10:07.971145 11575 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 02:10:07.971828 12050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 02:10:07.972901 12050 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: tablet copy: Starting download of 0 data blocks...
I20260809 02:10:07.973070 12050 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: tablet copy: Starting download of 1 WAL segments...
I20260809 02:10:07.974150 12050 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 02:10:07.975144 11575 tablet_copy_service.cc:342] P 86d06056e8354223aff7fe1f4a8cc361: Request end of tablet copy session 5fc6594b92bd463ca3ad7d723123c664-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.9.189@KRBTEST.COM'} at 127.0.0.1:49806
I20260809 02:10:07.975214 11575 tablet_copy_service.cc:433] P 86d06056e8354223aff7fe1f4a8cc361: ending tablet copy session 5fc6594b92bd463ca3ad7d723123c664-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5fc6594b92bd463ca3ad7d723123c664
I20260809 02:10:07.978133 12050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:07.979431 12163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:07.979451 12166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:07.979521 12164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:07.979835 12050 server_base.cc:1034] running on GCE node
I20260809 02:10:07.979972 12050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:07.980137 12050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:07.981304 12050 hybrid_clock.cc:648] HybridClock initialized: now 1786241407981295 us; error 27 us; skew 500 ppm
Aug 09 02:10:07 dist-test-slave-p3t5 krb5kdc[11504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241407, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:07.982903 12050 init.cc:377] Logged in from keytab as oryx/127.5.9.189@KRBTEST.COM (short username oryx)
I20260809 02:10:07.983265 12050 webserver.cc:460] Webserver started at http://127.5.9.189:40639/ using document root <none> and password file <none>
I20260809 02:10:07.983424 12050 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:07.983480 12050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:07.984190 12050 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:10:07.984836 12173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:07.984997 12050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:10:07.985051 12050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/wal
uuid: "5fc6594b92bd463ca3ad7d723123c664"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "c39dce0bf3e3e836b22d60a95245a6d9"
server_key_iv: "1ac75bd8bce15112b28289eeba8769d1"
server_key_version: "encryptionkey@0"
I20260809 02:10:07.985129 12050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:07.996663 12050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:07.998584 12050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:08.002095 12050 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.189:40825
I20260809 02:10:08.002115 12230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.189:40825 every 8 connection(s)
I20260809 02:10:08.002486 12050 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-1/data/info.pb
I20260809 02:10:08.003487 12231 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:10:08.004093 12231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: Bootstrap starting.
I20260809 02:10:08.007745 12231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:10:08.008026 12231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: Bootstrap complete.
I20260809 02:10:08.008252 12231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: NON_VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: true } }
I20260809 02:10:08.008355 12231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5fc6594b92bd463ca3ad7d723123c664, State: Initialized, Role: LEARNER
I20260809 02:10:08.008401 12231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: NON_VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: true } }
I20260809 02:10:08.008623 12234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: NON_VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: true } } }
I20260809 02:10:08.008636 12231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:08.008746 12234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: This master's current role is: LEARNER
I20260809 02:10:08.010316 12247 catalog_manager.cc:1259] Loaded cluster ID: 8c80a661add343e39c95dbfcedfb468e
I20260809 02:10:08.010528 12247 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: loading cluster ID for follower catalog manager: success
I20260809 02:10:08.011566 12247 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: acquiring CA information for follower catalog manager: success
I20260809 02:10:08.011812 12247 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:10:08.054037 12248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.046594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58998 (local address 127.5.9.189:40825)
0809 02:10:08.046809 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.046811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.046821 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:10:08.046848 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.046849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.046859 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.047014 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.047674 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.048382 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.049165 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.049398 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.049491 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.050959 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.050961 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.050961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.050970 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.052637 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.053050 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.053051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.053051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.053121 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.053334 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.053335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.053336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.053469 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.053578 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.053750 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.053845 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for oryx/127.5.9.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.9.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.9.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.188.keytab.
Entry for principal oryx/127.5.9.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.188.keytab.
I20260809 02:10:08.085848 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.188:45949
--webserver_interface=127.5.9.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.188.keytab
--principal=oryx/127.5.9.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:39773,127.5.9.189:40825,127.5.9.188:45949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:08.177596 12252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:08.177955 12252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:08.177984 12252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:08.179364 12252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:08.179417 12252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:08.179431 12252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:08.179445 12252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:08.179457 12252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:10:08.181304 12252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:39773,127.5.9.189:40825,127.5.9.188:45949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.188.keytab
--principal=oryx/127.5.9.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.188:45949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.9.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:08.181597 12252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:08.181937 12252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:08.187515 12261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:08.187588 12252 server_base.cc:1034] running on GCE node
W20260809 02:10:08.187525 12257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:08.187510 12258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:08.188066 12252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:08.188405 12252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:08.189559 12252 hybrid_clock.cc:648] HybridClock initialized: now 1786241408189546 us; error 31 us; skew 500 ppm
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:08.192859 12252 init.cc:377] Logged in from keytab as oryx/127.5.9.188@KRBTEST.COM (short username oryx)
I20260809 02:10:08.193450 12252 webserver.cc:460] Webserver started at http://127.5.9.188:38777/ using document root <none> and password file <none>
I20260809 02:10:08.193785 12252 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:08.193861 12252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:08.194016 12252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:08.195217 12252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/data/instance:
uuid: "6cdde3d6ae9041d99deeb13f382cabd6"
format_stamp: "Formatted at 2026-08-09 02:10:08 on dist-test-slave-p3t5"
server_key: "6f50a90e9ab24e8d11aba182027dea95"
server_key_iv: "40495673150895c682946e567e19a4e7"
server_key_version: "encryptionkey@0"
I20260809 02:10:08.195619 12252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/wal/instance:
uuid: "6cdde3d6ae9041d99deeb13f382cabd6"
format_stamp: "Formatted at 2026-08-09 02:10:08 on dist-test-slave-p3t5"
server_key: "6f50a90e9ab24e8d11aba182027dea95"
server_key_iv: "40495673150895c682946e567e19a4e7"
server_key_version: "encryptionkey@0"
I20260809 02:10:08.197082 12252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:10:08.198062 12268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:08.198295 12252 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:08.198405 12252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/wal
uuid: "6cdde3d6ae9041d99deeb13f382cabd6"
format_stamp: "Formatted at 2026-08-09 02:10:08 on dist-test-slave-p3t5"
server_key: "6f50a90e9ab24e8d11aba182027dea95"
server_key_iv: "40495673150895c682946e567e19a4e7"
server_key_version: "encryptionkey@0"
I20260809 02:10:08.198496 12252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:08.208446 12252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:08.211028 12252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:08.211222 12252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:08.216557 12252 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.188:45949
I20260809 02:10:08.216607 12323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.188:45949 every 8 connection(s)
I20260809 02:10:08.217031 12252 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/data/info.pb
I20260809 02:10:08.218263 12324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:08.220158 12324 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:10:08.223119 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 12252
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.188@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.9.188@KRBTEST.COM
I20260809 02:10:08.231990 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.221136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49808 (local address 127.5.9.190:39773)
0809 02:10:08.221201 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.221204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.221411 (+ 207us) server_negotiation.cc:407] Connection header received
0809 02:10:08.222120 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.222121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.222133 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.222205 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.222991 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.223717 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.224892 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.225104 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.225198 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.228268 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.228273 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.228281 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.228294 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.230082 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.230793 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.230794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.230794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.230856 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.231189 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.231190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.231190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.231325 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.231435 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.231778 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.231875 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:10:08.232187 12326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.220930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.0.0.1:49808)
0809 02:10:08.221236 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.221289 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.222017 (+ 728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.222289 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.222293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.222584 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.222871 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.222877 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.223845 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.223846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.224790 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.224793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.225209 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.225210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.225338 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.225979 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.225988 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.228029 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.230220 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.230229 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.230232 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.230665 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.230935 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.230937 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.230937 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.231085 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.231452 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.231459 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.231619 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.231740 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.231917 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":134,"threads_started":1}
I20260809 02:10:08.232853 12328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.223738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49284 (local address 127.5.9.188:45949)
0809 02:10:08.224005 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.224009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.224028 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:10:08.224047 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.224049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.224065 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.224157 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.224765 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.225493 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.226301 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.226619 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.226707 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.229216 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.229227 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.229228 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.229240 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.231192 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.231566 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.231569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.231569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.231634 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.231846 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.231847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.231848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.232031 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.232655 (+ 624us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.232710 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.232780 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":184,"threads_started":1}
I20260809 02:10:08.232903 12324 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.188@KRBTEST.COM for oryx/127.5.9.189@KRBTEST.COM
I20260809 02:10:08.239769 12326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.233219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:40825 (local address 127.0.0.1:59012)
0809 02:10:08.233275 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.233286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.233332 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.233472 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.233473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.233662 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.233707 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.233709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.234397 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.234398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.234907 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.234908 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.235307 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.235308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.235372 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.235803 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.235808 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.237054 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.238898 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.238899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.238900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.239074 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.239268 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.239268 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.239268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.239322 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.239517 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.239518 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.239559 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.239610 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.239695 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 02:10:08.239769 12248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.233289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59012 (local address 127.5.9.189:40825)
0809 02:10:08.233337 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.233338 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.233367 (+ 29us) server_negotiation.cc:407] Connection header received
0809 02:10:08.233394 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.233395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.233404 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.233468 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.233794 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.234300 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.234993 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.235211 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.235308 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.237192 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.237193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.237193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.237203 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.238799 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.239143 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.239144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.239144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.239200 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.239360 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.239361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.239361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.239466 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.239514 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.239626 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.239699 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:10:08.240298 12324 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.188@KRBTEST.COM for oryx/127.5.9.188@KRBTEST.COM
I20260809 02:10:08.248431 12326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.240677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:45949 (local address 127.0.0.1:49294)
0809 02:10:08.240740 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.240753 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.240800 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.241234 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.241236 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.241440 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.241532 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.241534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.242406 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.242407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.242973 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.242976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.243387 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.243388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.243450 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.244009 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.244014 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.245399 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.247160 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.247162 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.247163 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.247437 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.247769 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.247770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.247771 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.247846 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.248132 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.248133 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.248175 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.248229 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.248331 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 02:10:08.248522 12328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.240803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49294 (local address 127.5.9.188:45949)
0809 02:10:08.240894 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.240896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.240905 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:10:08.240922 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.240923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.240933 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.241007 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.241759 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.242299 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.243078 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.243306 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.243367 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.245501 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.245502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.245503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.245510 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.247033 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.247576 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.247577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.247578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.247657 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.247924 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.247925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.247925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.248062 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.248120 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.248248 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.248383 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":69}
I20260809 02:10:08.250084 12324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: Bootstrap starting.
I20260809 02:10:08.251024 12324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:08.251345 12324 log.cc:824] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:08.252305 12324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: No bootstrap required, opened a new log
I20260809 02:10:08.254202 12324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } }
I20260809 02:10:08.254364 12324 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:08.254388 12324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cdde3d6ae9041d99deeb13f382cabd6, State: Initialized, Role: FOLLOWER
I20260809 02:10:08.254524 12324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } }
I20260809 02:10:08.254881 12324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:08.254914 12330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } } }
I20260809 02:10:08.255131 12330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 02:10:08.257254 12341 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:10:08.257329 12341 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:10:08.260157 12252 master_runner.cc:428] Detected that this master 6cdde3d6ae9041d99deeb13f382cabd6 is joining an existing cluster
I20260809 02:10:08.260217 12252 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.9.188:45949
I20260809 02:10:08.269099 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.261462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49812 (local address 127.5.9.190:39773)
0809 02:10:08.261536 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.261539 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.261769 (+ 230us) server_negotiation.cc:407] Connection header received
0809 02:10:08.261848 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.261849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.261860 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.261911 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.262476 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.263247 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.264399 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.264703 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.264748 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.266136 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.266137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.266138 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.266147 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.267807 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.268176 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.268177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.268178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.268245 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.268447 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.268448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.268448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.268618 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.268693 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.268911 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.269000 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 02:10:08.269241 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.261435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.0.0.1:49812)
0809 02:10:08.261675 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.261687 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.261786 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.261970 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.261972 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.262219 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.262350 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.262353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.263342 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.263342 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.264301 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.264303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.264855 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.264855 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.264916 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.265512 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.265518 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.266030 (+ 512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.267903 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.267904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.267905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.268096 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.268316 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.268317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.268317 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.268381 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.268694 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.268695 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.268765 (+ 70us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.268847 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.268939 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":143,"threads_started":1}
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241407, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.190@KRBTEST.COM for oryx/127.5.9.188@KRBTEST.COM
I20260809 02:10:08.284657 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 11622
I20260809 02:10:08.292629 12328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.262233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49302 (local address 127.5.9.188:45949)
0809 02:10:08.262276 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.262278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.262548 (+ 270us) server_negotiation.cc:407] Connection header received
0809 02:10:08.262629 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.262630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.262638 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.262665 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.263166 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.263859 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.264935 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.265198 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.265238 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.288741 (+ 23503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.288742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.288743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.288749 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.291006 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.291840 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.291841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.291842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.291908 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.292091 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.292091 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.292092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.292222 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.292263 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.292432 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.292522 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:10:08.293157 12352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.261921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:45949 (local address 127.0.0.1:49302)
0809 02:10:08.262496 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.262505 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.262568 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.262716 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.262717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.263008 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.263075 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.263078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.263986 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.263986 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.264829 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.264831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.265311 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.265311 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.265395 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.287670 (+ 22275us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.287677 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.288136 (+ 459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.291107 (+ 2971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.291108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.291108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.291753 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.291975 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.291976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.291976 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.292029 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.292300 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.292301 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.292335 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.292759 (+ 424us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.292904 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":93,"threads_started":1}
I20260809 02:10:08.294871 12248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.262237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59024 (local address 127.5.9.189:40825)
0809 02:10:08.262286 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.262288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.262409 (+ 121us) server_negotiation.cc:407] Connection header received
0809 02:10:08.262457 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.262458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.262469 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.262522 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.263007 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.263725 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.264754 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.265093 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.265135 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.289066 (+ 23931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.289068 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.289068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.289076 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.292226 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.293870 (+ 1644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.293871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.293872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.293961 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.294184 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.294185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.294185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.294325 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.294378 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.294676 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.294771 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 02:10:08.295172 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.271636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:45949 (local address 127.0.0.1:49318)
0809 02:10:08.271691 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.271701 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.271743 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.272116 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.272117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.272306 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.272361 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.272363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.273061 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.273062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.273974 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.273978 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.274483 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.274484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.274535 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.275039 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.275044 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.276562 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.293887 (+ 17325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.293889 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.293890 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.294150 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.294368 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.294369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.294369 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.294434 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.294752 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.294753 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.294817 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.294871 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.295013 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 02:10:08.295241 12353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.261599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:40825 (local address 127.0.0.1:59024)
0809 02:10:08.262321 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.262333 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.262404 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.262571 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.262572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.262781 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.262899 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.262901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.263822 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.263823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.264665 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.264667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.265211 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.265212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.265258 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.288278 (+ 23020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.288283 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.288633 (+ 350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.293030 (+ 4397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.293031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.293031 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.293780 (+ 749us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.294046 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.294047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.294047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.294111 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.294411 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.294412 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.294454 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.294971 (+ 517us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.295083 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":171,"threads_started":1}
I20260809 02:10:08.295935 12354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.271723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49318 (local address 127.5.9.188:45949)
0809 02:10:08.271968 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.271969 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.271978 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:10:08.272011 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.272012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.272022 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.272091 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.272448 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.272973 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.274067 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.274405 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.274487 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.276655 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.276657 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.276657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.276665 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.293635 (+ 16970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.294234 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.294235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.294235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.294297 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.294505 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.294505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.294506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.294651 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.295629 (+ 978us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.295676 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.295785 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":136,"threads_started":1}
I20260809 02:10:08.296622 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:45209
--local_ip_for_outbound_sockets=127.5.9.129
--tserver_master_addrs=127.5.9.190:39773
--webserver_port=46347
--webserver_interface=127.5.9.129
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.129.keytab
--principal=oryx/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.9.190:39773,127.5.9.189:40825
--tserver_master_addrs=127.5.9.190:39773,127.5.9.189:40825,127.5.9.188:45949 with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:10:08.296617 11553 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.9.188:45949: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6"
member_type: NON_VOTER
last_known_addr {
host: "127.5.9.188"
port: 45949
}
attrs {
promote: true
}
}
dest_uuid: "86d06056e8354223aff7fe1f4a8cc361"
cas_config_opid_index: 5
I20260809 02:10:08.296903 11553 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: NON_VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: true } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } }
I20260809 02:10:08.297386 12151 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6cdde3d6ae9041d99deeb13f382cabd6 (127.5.9.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: NON_VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: true } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } } }
I20260809 02:10:08.297920 12298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:08.298440 12151 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.9.188:45949
I20260809 02:10:08.298516 12151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "86d06056e8354223aff7fe1f4a8cc361" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: NON_VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: true } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } } }
I20260809 02:10:08.298578 12151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:08.298801 12252 master.cc:537] Master@127.5.9.188:45949 shutting down...
I20260809 02:10:08.299041 12298 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 1 FOLLOWER]: Refusing update from remote peer 86d06056e8354223aff7fe1f4a8cc361: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 02:10:08.299712 12150 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:10:08.300796 11538 consensus_peers.cc:487] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 -> Peer 6cdde3d6ae9041d99deeb13f382cabd6 (127.5.9.188:45949): Couldn't send request to peer 6cdde3d6ae9041d99deeb13f382cabd6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 02:10:08.302003 12252 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:08.302107 12252 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:08.302160 12252 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: stopping tablet replica
I20260809 02:10:08.303795 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.297551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:40825 (local address 127.0.0.1:59032)
0809 02:10:08.299598 (+ 2047us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.299610 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.299665 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.299995 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.299996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.299997 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:08.300051 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.300053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.301049 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.301050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.302406 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.302409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.303468 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.303470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.303550 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.303584 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.303617 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.303676 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2026}
I20260809 02:10:08.304818 12248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.299391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59032 (local address 127.5.9.189:40825)
0809 02:10:08.299559 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.299561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.299721 (+ 160us) server_negotiation.cc:407] Connection header received
0809 02:10:08.299749 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.299750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.299775 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.299834 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:08.300135 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.300882 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.302507 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.303351 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.304547 (+ 1196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.304635 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.304656 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.304708 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260809 02:10:08.305193 12202 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 1 LEARNER]: Refusing update from remote peer 86d06056e8354223aff7fe1f4a8cc361: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 02:10:08.305419 12150 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: NON_VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 02:10:08.306239 12202 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6cdde3d6ae9041d99deeb13f382cabd6 (127.5.9.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: NON_VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: true } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } } }
I20260809 02:10:08.307715 12234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86d06056e8354223aff7fe1f4a8cc361. Latest consensus state: current_term: 1 leader_uuid: "86d06056e8354223aff7fe1f4a8cc361" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: NON_VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: true } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } } }
I20260809 02:10:08.307832 12234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: This master's current role is: LEARNER
I20260809 02:10:08.307873 12234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "86d06056e8354223aff7fe1f4a8cc361" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: NON_VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: true } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } } }
I20260809 02:10:08.307941 12234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: This master's current role is: LEARNER
I20260809 02:10:08.308341 12150 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361: attempting to promote NON_VOTER 5fc6594b92bd463ca3ad7d723123c664 to VOTER
I20260809 02:10:08.308629 12150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } }
I20260809 02:10:08.309434 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.309188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:45949 (local address 127.0.0.1:49322)
0809 02:10:08.309339 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.309389 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:45949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260809 02:10:08.309886 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.309782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:45949 (local address 127.0.0.1:49326)
0809 02:10:08.309826 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.309851 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:45949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 02:10:08.309973 12202 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 1 LEARNER]: Refusing update from remote peer 86d06056e8354223aff7fe1f4a8cc361: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260809 02:10:08.309996 11538 consensus_peers.cc:487] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 -> Peer 6cdde3d6ae9041d99deeb13f382cabd6 (127.5.9.188:45949): Couldn't send request to peer 6cdde3d6ae9041d99deeb13f382cabd6. Status: Network error: Client connection negotiation failed: client connection to 127.5.9.188:45949: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 02:10:08.310235 12151 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 02:10:08.311031 12152 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5fc6594b92bd463ca3ad7d723123c664 (127.5.9.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } } }
I20260809 02:10:08.312281 12152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "86d06056e8354223aff7fe1f4a8cc361" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } } }
I20260809 02:10:08.312381 12152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:08.312438 12152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "86d06056e8354223aff7fe1f4a8cc361" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } } }
I20260809 02:10:08.312498 12152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:08.312849 12202 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5fc6594b92bd463ca3ad7d723123c664 (127.5.9.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } } }
I20260809 02:10:08.314347 12357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "86d06056e8354223aff7fe1f4a8cc361" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } } }
I20260809 02:10:08.314472 12357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:10:08.315609 12252 master.cc:552] Master@127.5.9.188:45949 shutdown complete.
I20260809 02:10:08.315663 12252 master_runner.cc:305] Clearing existing system tablet
I20260809 02:10:08.315892 12252 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:08.315946 12252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:08.316740 12252 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 02:10:08.317335 12359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:08.317504 12252 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:08.317564 12252 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:10:08.317587 12252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/wal
uuid: "6cdde3d6ae9041d99deeb13f382cabd6"
format_stamp: "Formatted at 2026-08-09 02:10:08 on dist-test-slave-p3t5"
server_key: "6f50a90e9ab24e8d11aba182027dea95"
server_key_iv: "40495673150895c682946e567e19a4e7"
server_key_version: "encryptionkey@0"
I20260809 02:10:08.318461 12252 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:08.319898 12252 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 02:10:08.319972 12252 log.cc:1196] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 02:10:08.320182 12252 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: Deleting consensus metadata
I20260809 02:10:08.320302 12252 master_runner.cc:315] Copying system tablet from 127.5.9.190:39773
I20260809 02:10:08.320992 12252 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: tablet copy: Beginning tablet copy session from remote peer at address 127.5.9.190:39773
I20260809 02:10:08.329939 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.321964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49824 (local address 127.5.9.190:39773)
0809 02:10:08.322023 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.322026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.322266 (+ 240us) server_negotiation.cc:407] Connection header received
0809 02:10:08.322385 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.322386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.322399 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.322469 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.322931 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.323748 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.324915 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.325234 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.325277 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.326658 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.326660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.326660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.326668 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.328470 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.328923 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.328924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.328925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.328996 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.329225 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.329226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.329226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.329381 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.329453 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.329704 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.329818 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 02:10:08.329941 12365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.321878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.0.0.1:49824)
0809 02:10:08.322149 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.322192 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.322297 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.322490 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.322491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.322750 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.322830 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.322832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.323850 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.323851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.324801 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.324803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.325363 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.325364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.325435 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.326047 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.326054 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.326485 (+ 431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.328577 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.328579 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.328579 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.328829 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.329075 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.329076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.329076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.329153 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.329459 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.329460 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.329520 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.329601 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.329763 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":204,"threads_started":1}
I20260809 02:10:08.330700 11575 tablet_copy_service.cc:140] P 86d06056e8354223aff7fe1f4a8cc361: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6cdde3d6ae9041d99deeb13f382cabd6 ({username='oryx', principal='oryx/127.5.9.188@KRBTEST.COM'} at 127.0.0.1:49824)
I20260809 02:10:08.330778 11575 tablet_copy_service.cc:161] P 86d06056e8354223aff7fe1f4a8cc361: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6cdde3d6ae9041d99deeb13f382cabd6 at {username='oryx', principal='oryx/127.5.9.188@KRBTEST.COM'} at 127.0.0.1:49824: session id = 6cdde3d6ae9041d99deeb13f382cabd6-00000000000000000000000000000000
I20260809 02:10:08.331400 11575 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 02:10:08.332139 12252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 02:10:08.333525 12252 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: tablet copy: Starting download of 0 data blocks...
I20260809 02:10:08.333748 12252 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: tablet copy: Starting download of 1 WAL segments...
I20260809 02:10:08.334559 12252 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 02:10:08.335572 11575 tablet_copy_service.cc:342] P 86d06056e8354223aff7fe1f4a8cc361: Request end of tablet copy session 6cdde3d6ae9041d99deeb13f382cabd6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.9.188@KRBTEST.COM'} at 127.0.0.1:49824
I20260809 02:10:08.335637 11575 tablet_copy_service.cc:433] P 86d06056e8354223aff7fe1f4a8cc361: ending tablet copy session 6cdde3d6ae9041d99deeb13f382cabd6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6cdde3d6ae9041d99deeb13f382cabd6
I20260809 02:10:08.338199 12252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:08.339362 12366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:08.339370 12369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:08.339748 12367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:08.340101 12252 server_base.cc:1034] running on GCE node
I20260809 02:10:08.340199 12252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:08.340345 12252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:08.341475 12252 hybrid_clock.cc:648] HybridClock initialized: now 1786241408341449 us; error 40 us; skew 500 ppm
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:08.343226 12252 init.cc:377] Logged in from keytab as oryx/127.5.9.188@KRBTEST.COM (short username oryx)
I20260809 02:10:08.343525 12252 webserver.cc:460] Webserver started at http://127.5.9.188:34781/ using document root <none> and password file <none>
I20260809 02:10:08.343655 12252 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:08.343693 12252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:08.344494 12252 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:10:08.345166 12376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:08.345350 12252 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:08.345412 12252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/wal
uuid: "6cdde3d6ae9041d99deeb13f382cabd6"
format_stamp: "Formatted at 2026-08-09 02:10:08 on dist-test-slave-p3t5"
server_key: "6f50a90e9ab24e8d11aba182027dea95"
server_key_iv: "40495673150895c682946e567e19a4e7"
server_key_version: "encryptionkey@0"
I20260809 02:10:08.345479 12252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:08.365146 12252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:08.367228 12252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:08.370565 12252 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.188:45949
I20260809 02:10:08.370909 12433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.188:45949 every 8 connection(s)
I20260809 02:10:08.371019 12252 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/master-2/data/info.pb
I20260809 02:10:08.372007 12435 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:10:08.372527 12435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: Bootstrap starting.
I20260809 02:10:08.375689 12435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:10:08.375952 12435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: Bootstrap complete.
I20260809 02:10:08.376169 12435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } }
I20260809 02:10:08.376272 12435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6cdde3d6ae9041d99deeb13f382cabd6, State: Initialized, Role: LEARNER
I20260809 02:10:08.376312 12435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } }
I20260809 02:10:08.376509 12435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:08.376585 12438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } } }
I20260809 02:10:08.376765 12438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [sys.catalog]: This master's current role is: LEARNER
I20260809 02:10:08.382058 12450 catalog_manager.cc:1259] Loaded cluster ID: 8c80a661add343e39c95dbfcedfb468e
I20260809 02:10:08.382143 12450 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: loading cluster ID for follower catalog manager: success
I20260809 02:10:08.383273 12450 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: acquiring CA information for follower catalog manager: success
I20260809 02:10:08.383643 12450 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 02:10:08.411088 12355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:08.411332 12355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:08.411361 12355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:08.412712 12355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:08.412806 12355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:08.412858 12355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:10:08.414690 12355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.129.keytab
--principal=oryx/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:45209
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=46347
--tserver_master_addrs=127.5.9.190:39773,127.5.9.189:40825,127.5.9.188:45949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:08.415026 12355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:08.415294 12355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:08.420662 12456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:08.420687 12455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:08.420679 12458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:08.421129 12355 server_base.cc:1034] running on GCE node
I20260809 02:10:08.421437 12355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:08.421751 12355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:08.422925 12355 hybrid_clock.cc:648] HybridClock initialized: now 1786241408422911 us; error 30 us; skew 500 ppm
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:08.426321 12355 init.cc:377] Logged in from keytab as oryx/127.5.9.129@KRBTEST.COM (short username oryx)
I20260809 02:10:08.426971 12355 webserver.cc:460] Webserver started at http://127.5.9.129:46347/ using document root <none> and password file <none>
I20260809 02:10:08.427238 12355 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:08.427341 12355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:08.428905 12355 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:08.429689 12465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:08.429878 12355 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:08.429979 12355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "3955f153a3fb49cfb79a720b1ae457b5"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "e605fa0e34927605b3e0be77dd160cef"
server_key_iv: "11753db9e7caa024ebbd1eb74ab49f3b"
server_key_version: "encryptionkey@0"
I20260809 02:10:08.430348 12355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:08.450230 12355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:08.452477 12355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:08.452611 12355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:08.452818 12355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:08.453195 12355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:08.453239 12355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:08.453264 12355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:08.453301 12355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:08.460265 12355 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:45209
I20260809 02:10:08.460336 12583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:45209 every 8 connection(s)
I20260809 02:10:08.460743 12355 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.188@KRBTEST.COM
I20260809 02:10:08.469180 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 12355
I20260809 02:10:08.469357 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 11767
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.189@KRBTEST.COM
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM
I20260809 02:10:08.474707 12248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.462086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:47191 (local address 127.5.9.189:40825)
0809 02:10:08.462139 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.462141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.462207 (+ 66us) server_negotiation.cc:407] Connection header received
0809 02:10:08.463405 (+ 1198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.463406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.463415 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.463478 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.464254 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.464765 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.466297 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.466597 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.466641 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.470508 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.470510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.470510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.470521 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.472594 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.473172 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.473174 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.473174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.473245 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.473644 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.473644 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.473645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.473813 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.473879 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.474469 (+ 590us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.474563 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 02:10:08.475025 12591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.462009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:40825 (local address 127.5.9.129:47191)
0809 02:10:08.462147 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.462157 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.463077 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.463486 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.463487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.463899 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.464149 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.464155 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.464878 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.464880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.466184 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.466187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.466754 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.466756 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.466830 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.467543 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.467555 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.470367 (+ 2812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.472725 (+ 2358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.472736 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.472740 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.473080 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.473333 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.473334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.473335 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.473541 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.474013 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.474022 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.474268 (+ 246us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.474451 (+ 183us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.474702 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":32,"thread_start_us":78,"threads_started":1}
I20260809 02:10:08.475639 12586 heartbeater.cc:347] Connected to a master server at 127.5.9.189:40825
I20260809 02:10:08.475769 12586 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:08.476063 12586 heartbeater.cc:511] Master 127.5.9.189:40825 requested a full tablet report, sending...
I20260809 02:10:08.476688 12191 ts_manager.cc:194] Registered new tserver with Master: 3955f153a3fb49cfb79a720b1ae457b5 (127.5.9.129:45209)
I20260809 02:10:08.480123 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.461593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.129:49897)
0809 02:10:08.461864 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.461876 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.463072 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.463373 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.463378 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.463905 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.464151 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.464157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.465136 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.465137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.466184 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.466187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.466756 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.466757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.466847 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.467582 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.467587 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.476588 (+ 9001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.479056 (+ 2468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.479057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.479058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.479241 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.479488 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.479489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.479489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.479541 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.479843 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.479844 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.479890 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.479936 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.480032 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":15,"thread_start_us":104,"threads_started":1}
I20260809 02:10:08.480245 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.461591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:49897 (local address 127.5.9.190:39773)
0809 02:10:08.461708 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.461711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.461942 (+ 231us) server_negotiation.cc:407] Connection header received
0809 02:10:08.463192 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.463194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.463209 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.463267 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.464267 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.465051 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.466297 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.466612 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.466651 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.476713 (+ 10062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.476716 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.476716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.476727 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.478950 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.479342 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.479343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.479343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.479418 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.479612 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.479613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.479614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.479783 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.479852 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.479998 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.480096 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 02:10:08.480633 12585 heartbeater.cc:347] Connected to a master server at 127.5.9.190:39773
I20260809 02:10:08.480693 12585 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:08.480786 12585 heartbeater.cc:511] Master 127.5.9.190:39773 requested a full tablet report, sending...
I20260809 02:10:08.481118 11553 ts_manager.cc:194] Re-registered known tserver with Master: 3955f153a3fb49cfb79a720b1ae457b5 (127.5.9.129:45209)
I20260809 02:10:08.481662 11553 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:49897
I20260809 02:10:08.483338 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:46733
--local_ip_for_outbound_sockets=127.5.9.130
--tserver_master_addrs=127.5.9.190:39773
--webserver_port=42097
--webserver_interface=127.5.9.130
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.130.keytab
--principal=oryx/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.9.190:39773,127.5.9.189:40825
--tserver_master_addrs=127.5.9.190:39773,127.5.9.189:40825,127.5.9.188:45949 with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:10:08.485841 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.461685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:46551 (local address 127.5.9.188:45949)
0809 02:10:08.461945 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.461946 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.461956 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:10:08.463261 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.463263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.463275 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.463338 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.464267 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.465247 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.466290 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.466520 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.466610 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.482364 (+ 15754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.482366 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.482366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.482377 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.484327 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.484872 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.484873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.484874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.484951 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.485196 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.485197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.485197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.485337 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.485417 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.485605 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.485728 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":55,"threads_started":1}
I20260809 02:10:08.486086 12588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.461593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:45949 (local address 127.5.9.129:46551)
0809 02:10:08.461800 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.461847 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.463065 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.463373 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.463378 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.463898 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.464151 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.464157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.465328 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.465329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.466179 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.466182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.466623 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.466624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.466787 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.467663 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.467668 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.482247 (+ 14579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.484443 (+ 2196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.484445 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.484446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.484686 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.485044 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.485045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.485046 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.485115 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.485410 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.485411 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.485463 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.485562 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.485727 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":31,"thread_start_us":130,"threads_started":1}
I20260809 02:10:08.486634 12584 heartbeater.cc:347] Connected to a master server at 127.5.9.188:45949
I20260809 02:10:08.486694 12584 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:08.486821 12584 heartbeater.cc:511] Master 127.5.9.188:45949 requested a full tablet report, sending...
I20260809 02:10:08.487324 12394 ts_manager.cc:194] Registered new tserver with Master: 3955f153a3fb49cfb79a720b1ae457b5 (127.5.9.129:45209)
W20260809 02:10:08.576973 12592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:08.577262 12592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:08.577312 12592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:08.578780 12592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:08.578859 12592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:08.578907 12592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:10:08.580499 12592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.130.keytab
--principal=oryx/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:46733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=42097
--tserver_master_addrs=127.5.9.190:39773,127.5.9.189:40825,127.5.9.188:45949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:08.580816 12592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:08.581094 12592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:08.587018 12601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:08.587059 12598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:08.587024 12599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:08.587287 12592 server_base.cc:1034] running on GCE node
I20260809 02:10:08.587548 12592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:08.587870 12592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:08.589061 12592 hybrid_clock.cc:648] HybridClock initialized: now 1786241408589053 us; error 47 us; skew 500 ppm
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:08.592253 12592 init.cc:377] Logged in from keytab as oryx/127.5.9.130@KRBTEST.COM (short username oryx)
I20260809 02:10:08.592856 12592 webserver.cc:460] Webserver started at http://127.5.9.130:42097/ using document root <none> and password file <none>
I20260809 02:10:08.593142 12592 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:08.593235 12592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:08.594830 12592 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:08.595510 12608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:08.595724 12592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:08.595849 12592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "b2976b42ac924f29bc197e23f317edb4"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "ff7ff0b01da4b1c02396a825f39d7a6f"
server_key_iv: "468bc77b22d63d70193b6d58a0e90d8e"
server_key_version: "encryptionkey@0"
I20260809 02:10:08.596278 12592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:08.617063 12592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:08.619918 12592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:08.620139 12592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:08.620431 12592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:08.620788 12592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:08.620851 12592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:08.620931 12592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:08.620962 12592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:08.628259 12592 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:46733
I20260809 02:10:08.628314 12733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:46733 every 8 connection(s)
I20260809 02:10:08.628731 12592 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:10:08.630865 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 12592
I20260809 02:10:08.631084 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 11908
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.189@KRBTEST.COM
I20260809 02:10:08.641568 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.629605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:44695 (local address 127.5.9.190:39773)
0809 02:10:08.629691 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.629694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.629999 (+ 305us) server_negotiation.cc:407] Connection header received
0809 02:10:08.631211 (+ 1212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.631213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.631226 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.631255 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.632474 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.633006 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.634576 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.634903 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.634949 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.637757 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.637758 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.637759 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.637768 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.639639 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.640188 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.640190 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.640190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.640296 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.640813 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.640814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.640815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.640948 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.641036 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.641375 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.641465 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:10:08.641813 12739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.629626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.130:44695)
0809 02:10:08.629902 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.629914 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.631085 (+ 1171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.631438 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.631443 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.632052 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.632313 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.632318 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.633153 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.633154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.634461 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.634465 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.635055 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.635057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.635229 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.635882 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.635893 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.637610 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.639742 (+ 2132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.639752 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.639755 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.640092 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.640392 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.640393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.640393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.640507 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.641027 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.641035 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.641226 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.641340 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.641533 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":3,"thread_start_us":192,"threads_started":1}
I20260809 02:10:08.642407 12735 heartbeater.cc:347] Connected to a master server at 127.5.9.190:39773
I20260809 02:10:08.642544 12735 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:08.642768 12735 heartbeater.cc:511] Master 127.5.9.190:39773 requested a full tablet report, sending...
I20260809 02:10:08.643200 11553 ts_manager.cc:194] Re-registered known tserver with Master: b2976b42ac924f29bc197e23f317edb4 (127.5.9.130:46733)
I20260809 02:10:08.643424 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:35771
--local_ip_for_outbound_sockets=127.5.9.131
--tserver_master_addrs=127.5.9.190:39773
--webserver_port=42315
--webserver_interface=127.5.9.131
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.131.keytab
--principal=oryx/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.9.190:39773,127.5.9.189:40825
--tserver_master_addrs=127.5.9.190:39773,127.5.9.189:40825,127.5.9.188:45949 with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:10:08.643719 11553 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:44695
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.188@KRBTEST.COM
I20260809 02:10:08.646510 12740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.630132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:40825 (local address 127.5.9.130:55129)
0809 02:10:08.630252 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.630262 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.634900 (+ 4638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.635143 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.635144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.635317 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.635390 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.635391 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.636362 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.636364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.637339 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.637342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.637876 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.637877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.637965 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.638584 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.638590 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.640199 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.642471 (+ 2272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.642472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.642472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.642707 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.642944 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.642945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.642945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.643019 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.646097 (+ 3078us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.646098 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.646174 (+ 76us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.646267 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.646386 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":73,"threads_started":1}
I20260809 02:10:08.646510 12248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.630229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:55129 (local address 127.5.9.189:40825)
0809 02:10:08.630275 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.630277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.630309 (+ 32us) server_negotiation.cc:407] Connection header received
0809 02:10:08.635006 (+ 4697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.635008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.635026 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.635061 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.635513 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.636251 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.637456 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.637791 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.637870 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.640331 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.640333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.640333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.640342 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.642020 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.642786 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.642787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.642788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.642859 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.645413 (+ 2554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.645415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.645415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.645582 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.646104 (+ 522us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.646300 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.646421 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:10:08.646919 12736 heartbeater.cc:347] Connected to a master server at 127.5.9.189:40825
I20260809 02:10:08.646986 12736 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:08.647090 12736 heartbeater.cc:511] Master 127.5.9.189:40825 requested a full tablet report, sending...
I20260809 02:10:08.647480 12191 ts_manager.cc:194] Registered new tserver with Master: b2976b42ac924f29bc197e23f317edb4 (127.5.9.130:46733)
I20260809 02:10:08.648800 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.629965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:55115 (local address 127.5.9.188:45949)
0809 02:10:08.630265 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.630267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.630278 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:10:08.631242 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.631243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.631259 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.631290 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.640250 (+ 8960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.641049 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.642328 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.642644 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.642695 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.644989 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.644990 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.644991 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.645001 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.647498 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.647917 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.647918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.647918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.647989 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.648223 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.648224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.648224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.648349 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.648433 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.648594 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.648694 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271}
I20260809 02:10:08.649075 12738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.629626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:45949 (local address 127.5.9.130:55115)
0809 02:10:08.629857 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.629906 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.631092 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.631439 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.631443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.634728 (+ 3285us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.634822 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.634824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.641195 (+ 6371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.641196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.642237 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.642240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.642779 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.642781 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.642856 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.643285 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.643290 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.644831 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.647593 (+ 2762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.647595 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.647595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.647818 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.648076 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.648077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.648077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.648148 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.648425 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.648426 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.648473 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.648556 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.648694 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":120,"threads_started":1}
I20260809 02:10:08.649580 12734 heartbeater.cc:347] Connected to a master server at 127.5.9.188:45949
I20260809 02:10:08.649677 12734 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:08.649741 12734 heartbeater.cc:511] Master 127.5.9.188:45949 requested a full tablet report, sending...
I20260809 02:10:08.650089 12394 ts_manager.cc:194] Registered new tserver with Master: b2976b42ac924f29bc197e23f317edb4 (127.5.9.130:46733)
W20260809 02:10:08.735252 12741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:08.735584 12741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:08.735632 12741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:08.736931 12741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:08.737010 12741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:08.737079 12741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:10:08.738803 12741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.131.keytab
--principal=oryx/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:35771
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=42315
--tserver_master_addrs=127.5.9.190:39773,127.5.9.189:40825,127.5.9.188:45949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:08.739224 12741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:08.739519 12741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:08.745491 12746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:08.745498 12749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:08.745507 12747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:08.745712 12741 server_base.cc:1034] running on GCE node
I20260809 02:10:08.746085 12741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:08.746413 12741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:08.747604 12741 hybrid_clock.cc:648] HybridClock initialized: now 1786241408747614 us; error 70 us; skew 500 ppm
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:08.750795 12741 init.cc:377] Logged in from keytab as oryx/127.5.9.131@KRBTEST.COM (short username oryx)
I20260809 02:10:08.751329 12741 webserver.cc:460] Webserver started at http://127.5.9.131:42315/ using document root <none> and password file <none>
I20260809 02:10:08.751617 12741 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:08.751715 12741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:08.753233 12741 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:10:08.754031 12757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:08.754334 12741 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:08.754478 12741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "d811603a56444215834731cbff681ca1"
format_stamp: "Formatted at 2026-08-09 02:10:07 on dist-test-slave-p3t5"
server_key: "c518afda707c5c7ced1d34218ebbe91f"
server_key_iv: "182c6aa6e8d9d2c3af4fec3a9bb075b2"
server_key_version: "encryptionkey@0"
I20260809 02:10:08.754913 12741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:08.765846 12741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:08.768708 12741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:08.768983 12741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:08.769296 12741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:08.769747 12741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:08.769815 12741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:08.769866 12741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:08.769897 12741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:08.776767 12741 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:35771
I20260809 02:10:08.776835 12876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:35771 every 8 connection(s)
I20260809 02:10:08.777392 12741 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:10:08.781056 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 12741
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.188@KRBTEST.COM
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM
Aug 09 02:10:08 dist-test-slave-p3t5 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241408, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.189@KRBTEST.COM
I20260809 02:10:08.793279 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.779081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:45783 (local address 127.5.9.188:45949)
0809 02:10:08.779137 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.779138 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.780625 (+ 1487us) server_negotiation.cc:407] Connection header received
0809 02:10:08.780656 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.780657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.780671 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.780729 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.782252 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.783056 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.785327 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.785804 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.785877 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.789197 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.789199 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.789199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.789209 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.791391 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.792012 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.792013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.792014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.792086 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.792393 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.792394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.792395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.792533 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.792619 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.793032 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.793137 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 02:10:08.794049 12881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.778943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:45949 (local address 127.5.9.131:45783)
0809 02:10:08.780533 (+ 1590us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.780546 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.780601 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.780794 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.780795 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.781876 (+ 1081us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.781997 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.782003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.783182 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.783183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.785150 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.785155 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.785989 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.785989 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.786060 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.787011 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.787017 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.789043 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.791492 (+ 2449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.791503 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.791506 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.791912 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.792163 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.792164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.792164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.792314 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.792611 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.792619 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.792831 (+ 212us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.793449 (+ 618us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.793722 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1569,"mutex_wait_us":415,"thread_start_us":70,"threads_started":1}
I20260809 02:10:08.794657 12877 heartbeater.cc:347] Connected to a master server at 127.5.9.188:45949
I20260809 02:10:08.794780 12877 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:08.794763 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.778544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:52435 (local address 127.5.9.190:39773)
0809 02:10:08.778606 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.778609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.779271 (+ 662us) server_negotiation.cc:407] Connection header received
0809 02:10:08.780438 (+ 1167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.780439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.780459 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.780531 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.781879 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.782831 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.785184 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.785538 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.785595 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.789642 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.789644 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.789645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.789654 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.792243 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.793071 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.793072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.793072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.793156 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.793813 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.793814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.793815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.793972 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.794020 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.794545 (+ 525us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.794656 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 02:10:08.795017 12877 heartbeater.cc:511] Master 127.5.9.188:45949 requested a full tablet report, sending...
I20260809 02:10:08.795138 12882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.778568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:40825 (local address 127.5.9.131:50305)
0809 02:10:08.779090 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.779147 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.780305 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.780702 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.780703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.781301 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.781721 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.781730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.783034 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.783036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.785160 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.785162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.785882 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.785884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.785990 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.786872 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.786878 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.790413 (+ 3535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.793845 (+ 3432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.793846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.793846 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.794076 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.794339 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.794340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.794340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.794415 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.794772 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.794773 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.794841 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.794917 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.795041 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":198,"thread_start_us":104,"threads_started":1}
I20260809 02:10:08.795450 12248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.778633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:50305 (local address 127.5.9.189:40825)
0809 02:10:08.778679 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.778681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.779248 (+ 567us) server_negotiation.cc:407] Connection header received
0809 02:10:08.780618 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.780620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.780634 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.780682 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.781879 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.782820 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.785346 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.785738 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.785811 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.790519 (+ 4708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.790521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.790521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.790531 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.793739 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.794172 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.794174 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.794174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.794249 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.794496 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.794497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.794497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.794652 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.794739 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.795257 (+ 518us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.795364 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:10:08.795552 12394 ts_manager.cc:194] Registered new tserver with Master: d811603a56444215834731cbff681ca1 (127.5.9.131:35771)
I20260809 02:10:08.795913 12879 heartbeater.cc:347] Connected to a master server at 127.5.9.189:40825
I20260809 02:10:08.795987 12879 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:08.796069 12879 heartbeater.cc:511] Master 127.5.9.189:40825 requested a full tablet report, sending...
I20260809 02:10:08.796157 12884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.778568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.131:52435)
0809 02:10:08.779090 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.779147 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.780306 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.780629 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.780634 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.781375 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:08.781721 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.781730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.783034 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.783036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.785052 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.785058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.785765 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.785766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.785976 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.786684 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:08.786699 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:08.789500 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:08.792728 (+ 3228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.792729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.792730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.792976 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.793285 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:08.793286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:08.793286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:08.793698 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:08.794072 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:08.794073 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:08.794396 (+ 323us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.795357 (+ 961us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.795879 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":158,"thread_start_us":111,"threads_started":1}
I20260809 02:10:08.796509 12191 ts_manager.cc:194] Registered new tserver with Master: d811603a56444215834731cbff681ca1 (127.5.9.131:35771)
I20260809 02:10:08.796813 12878 heartbeater.cc:347] Connected to a master server at 127.5.9.190:39773
I20260809 02:10:08.796868 12878 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:08.796936 12878 heartbeater.cc:511] Master 127.5.9.190:39773 requested a full tablet report, sending...
I20260809 02:10:08.797199 11553 ts_manager.cc:194] Re-registered known tserver with Master: d811603a56444215834731cbff681ca1 (127.5.9.131:35771)
I20260809 02:10:08.797752 11553 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:52435
I20260809 02:10:08.803807 12895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.785121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49334 (local address 127.5.9.188:45949)
0809 02:10:08.785424 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.785426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.787498 (+ 2072us) server_negotiation.cc:407] Connection header received
0809 02:10:08.787528 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.787529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.787543 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.787605 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.790514 (+ 2909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.791456 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.797788 (+ 6332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.798325 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.798431 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.800153 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.800154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.800154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.800161 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.802099 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.802587 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.802588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.802589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.802652 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.802897 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.802898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.802898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.803061 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.803158 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.803550 (+ 392us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.803678 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":82,"threads_started":1}
I20260809 02:10:08.803874 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.799540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:45949 (local address 127.0.0.1:49350)
0809 02:10:08.799600 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.799613 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.799655 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.799879 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.799881 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.799882 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:08.799931 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.799933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.800962 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.800964 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.802394 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.802398 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.803472 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.803473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.803555 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.803559 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.803666 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.803723 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 02:10:08.804061 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.784393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49838 (local address 127.5.9.190:39773)
0809 02:10:08.785766 (+ 1373us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.785768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.785776 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:10:08.785799 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.785800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.785812 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.785883 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.787685 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.788438 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.794954 (+ 6516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.795472 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.795552 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.800074 (+ 4522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.800076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.800076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.800084 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.802379 (+ 2295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.802774 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.802775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.802776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.802836 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.803065 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.803066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.803066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.803202 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.803283 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.803576 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.803698 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1341,"thread_start_us":109,"threads_started":1}
I20260809 02:10:08.804183 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.799659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49350 (local address 127.5.9.188:45949)
0809 02:10:08.799702 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.799704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.799711 (+ 7us) server_negotiation.cc:407] Connection header received
0809 02:10:08.799754 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.799755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.799794 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.799876 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:08.799991 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.800849 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.802518 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.803264 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.803335 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.803432 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.804053 (+ 621us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.804098 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:10:08.804430 12406 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 02:10:08.804592 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.783965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59038 (local address 127.5.9.189:40825)
0809 02:10:08.785693 (+ 1728us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.785695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.787175 (+ 1480us) server_negotiation.cc:407] Connection header received
0809 02:10:08.787208 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.787209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.787225 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.787301 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.789245 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.793179 (+ 3934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.796425 (+ 3246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.797726 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.797802 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.800008 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.800010 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.800010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.800019 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.802587 (+ 2568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.803051 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.803052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.803052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.803123 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.803354 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.803355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.803355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.803764 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.803861 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.804377 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.804483 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1703,"thread_start_us":98,"threads_started":1}
I20260809 02:10:08.804637 12438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86d06056e8354223aff7fe1f4a8cc361. Latest consensus state: current_term: 1 leader_uuid: "86d06056e8354223aff7fe1f4a8cc361" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } } }
I20260809 02:10:08.804699 12438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [sys.catalog]: This master's current role is: LEARNER
I20260809 02:10:08.806453 11553 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:10:08.808269 11553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:10:08.809088 12151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "86d06056e8354223aff7fe1f4a8cc361" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: NON_VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: true } } }
I20260809 02:10:08.809203 12151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:08.810731 12152 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361: attempting to promote NON_VOTER 6cdde3d6ae9041d99deeb13f382cabd6 to VOTER
I20260809 02:10:08.810881 12152 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } }
I20260809 02:10:08.811529 12406 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 1 LEARNER]: Refusing update from remote peer 86d06056e8354223aff7fe1f4a8cc361: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 02:10:08.811805 12202 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 1 FOLLOWER]: Refusing update from remote peer 86d06056e8354223aff7fe1f4a8cc361: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 02:10:08.811929 12151 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 02:10:08.812214 12151 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 02:10:08.812796 12898 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6cdde3d6ae9041d99deeb13f382cabd6 (127.5.9.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } } }
I20260809 02:10:08.813180 12202 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6cdde3d6ae9041d99deeb13f382cabd6 (127.5.9.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } } }
I20260809 02:10:08.813736 12406 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6cdde3d6ae9041d99deeb13f382cabd6 (127.5.9.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } } }
I20260809 02:10:08.814074 12898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "86d06056e8354223aff7fe1f4a8cc361" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } } }
I20260809 02:10:08.814167 12898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:08.814221 12898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "86d06056e8354223aff7fe1f4a8cc361" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } } }
I20260809 02:10:08.814260 12898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:08.814296 12898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "86d06056e8354223aff7fe1f4a8cc361" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } } }
I20260809 02:10:08.814332 12898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:08.814426 12357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "86d06056e8354223aff7fe1f4a8cc361" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } } }
I20260809 02:10:08.814504 12357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:10:08.815860 12438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "86d06056e8354223aff7fe1f4a8cc361" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } } }
I20260809 02:10:08.815979 12438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:10:08.819373 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.815462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59550 (local address 127.5.9.129:45209)
0809 02:10:08.815932 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.815936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.815948 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:10:08.816012 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.816015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.816033 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.816114 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:08.816216 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.816907 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.818316 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.818998 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.819092 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.819181 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.819239 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.819288 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":222,"threads_started":1}
I20260809 02:10:08.819358 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.815224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:45209 (local address 127.0.0.1:59550)
0809 02:10:08.815463 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.815475 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.815522 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.816123 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.816124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.816126 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:08.816161 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.816163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.817000 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.817001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.818200 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.818202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.819090 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.819091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.819154 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.819158 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.819219 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.819268 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":86,"threads_started":1}
I20260809 02:10:08.819969 12906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.814654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:35771 (local address 127.0.0.1:50896)
0809 02:10:08.815355 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.815367 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.815464 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.815708 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.815709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.815711 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:08.815784 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.815786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.817422 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.817423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.818546 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.818548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.819687 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.819689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.819763 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.819776 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.819844 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.819890 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678}
I20260809 02:10:08.820391 12907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.815303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:46733 (local address 127.0.0.1:58590)
0809 02:10:08.816218 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.816229 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.816281 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.816706 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.816707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.816710 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:08.816756 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.816757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.818085 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.818086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.819140 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.819143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.820171 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.820172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.820229 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.820234 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.820278 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.820321 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"thread_start_us":116,"threads_started":1}
I20260809 02:10:08.820035 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.814842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50896 (local address 127.5.9.131:35771)
0809 02:10:08.815047 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.815053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.815428 (+ 375us) server_negotiation.cc:407] Connection header received
0809 02:10:08.815562 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.815567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.815588 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.815711 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:08.816062 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.817326 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.818646 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.819601 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.819676 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.819751 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.819876 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.819931 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":134,"threads_started":1}
I20260809 02:10:08.820583 12909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.815627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58590 (local address 127.5.9.130:46733)
0809 02:10:08.816476 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.816481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.816492 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:10:08.816533 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.816535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.816551 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.816630 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:08.816837 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.817985 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.819254 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.820062 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.820172 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.820263 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.820296 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.820349 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":101,"threads_started":1}
I20260809 02:10:08.820478 12515 tablet_service.cc:1467] Processing CreateTablet for tablet ec68375b89c9419790da1e6dddebfd7f (DEFAULT_TABLE table=test-table [id=e0a697459e3f470c8c805a5d9e215791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:08.820904 12515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec68375b89c9419790da1e6dddebfd7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:08.821206 12662 tablet_service.cc:1467] Processing CreateTablet for tablet ec68375b89c9419790da1e6dddebfd7f (DEFAULT_TABLE table=test-table [id=e0a697459e3f470c8c805a5d9e215791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:08.821494 12807 tablet_service.cc:1467] Processing CreateTablet for tablet ec68375b89c9419790da1e6dddebfd7f (DEFAULT_TABLE table=test-table [id=e0a697459e3f470c8c805a5d9e215791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:08.821699 12662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec68375b89c9419790da1e6dddebfd7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:08.821899 12807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec68375b89c9419790da1e6dddebfd7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:08.824579 12910 tablet_bootstrap.cc:492] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5: Bootstrap starting.
I20260809 02:10:08.825330 12910 tablet_bootstrap.cc:654] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:08.825712 12910 log.cc:824] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:08.826128 12911 tablet_bootstrap.cc:492] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4: Bootstrap starting.
I20260809 02:10:08.826946 12911 tablet_bootstrap.cc:654] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:08.827310 12911 log.cc:824] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:08.827720 12912 tablet_bootstrap.cc:492] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1: Bootstrap starting.
I20260809 02:10:08.828308 12912 tablet_bootstrap.cc:654] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:08.828357 12911 tablet_bootstrap.cc:492] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4: No bootstrap required, opened a new log
I20260809 02:10:08.828444 12911 ts_tablet_manager.cc:1397] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:10:08.828688 12912 log.cc:824] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:08.829787 12912 tablet_bootstrap.cc:492] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1: No bootstrap required, opened a new log
I20260809 02:10:08.829869 12912 ts_tablet_manager.cc:1397] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:10:08.830197 12910 tablet_bootstrap.cc:492] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5: No bootstrap required, opened a new log
I20260809 02:10:08.830307 12910 ts_tablet_manager.cc:1397] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260809 02:10:08.831221 12911 raft_consensus.cc:348] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:08.831427 12911 raft_consensus.cc:374] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:08.831470 12911 raft_consensus.cc:729] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2976b42ac924f29bc197e23f317edb4, State: Initialized, Role: FOLLOWER
I20260809 02:10:08.831614 12911 consensus_queue.cc:260] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:08.831925 12911 ts_tablet_manager.cc:1428] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 02:10:08.832037 12735 heartbeater.cc:503] Master 127.5.9.190:39773 was elected leader, sending a full tablet report...
I20260809 02:10:08.832443 12912 raft_consensus.cc:348] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:08.832610 12912 raft_consensus.cc:374] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:08.832655 12912 raft_consensus.cc:729] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d811603a56444215834731cbff681ca1, State: Initialized, Role: FOLLOWER
I20260809 02:10:08.832813 12912 consensus_queue.cc:260] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:08.833413 12912 ts_tablet_manager.cc:1428] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:10:08.833330 12910 raft_consensus.cc:348] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:08.833521 12910 raft_consensus.cc:374] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:08.833581 12910 raft_consensus.cc:729] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3955f153a3fb49cfb79a720b1ae457b5, State: Initialized, Role: FOLLOWER
I20260809 02:10:08.833758 12910 consensus_queue.cc:260] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:08.833801 12878 heartbeater.cc:503] Master 127.5.9.190:39773 was elected leader, sending a full tablet report...
I20260809 02:10:08.834267 12910 ts_tablet_manager.cc:1428] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260809 02:10:08.834684 12917 raft_consensus.cc:482] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:10:08.834792 12917 raft_consensus.cc:504] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:08.835062 12585 heartbeater.cc:503] Master 127.5.9.190:39773 was elected leader, sending a full tablet report...
I20260809 02:10:08.836019 12917 leader_election.cc:288] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d811603a56444215834731cbff681ca1 (127.5.9.131:35771), 3955f153a3fb49cfb79a720b1ae457b5 (127.5.9.129:45209)
I20260809 02:10:08.838524 12738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.835665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:35771 (local address 127.5.9.130:56537)
0809 02:10:08.835723 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.835735 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.835768 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.835988 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.835989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.835992 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:08.836086 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.836089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.836808 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.836809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.837594 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.837596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.838223 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.838224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.838317 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.838320 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.838401 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.838449 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 02:10:08.838524 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.835703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:56537 (local address 127.5.9.131:35771)
0809 02:10:08.835746 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.835748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.835774 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:10:08.835870 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.835871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.835884 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.835945 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:08.836174 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.836714 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.837749 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.838136 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.838224 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.838235 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.838410 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.838450 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:10:08.838886 12828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec68375b89c9419790da1e6dddebfd7f" candidate_uuid: "b2976b42ac924f29bc197e23f317edb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811603a56444215834731cbff681ca1" is_pre_election: true
I20260809 02:10:08.839036 12828 raft_consensus.cc:2454] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2976b42ac924f29bc197e23f317edb4 in term 0.
I20260809 02:10:08.839277 12612 leader_election.cc:302] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2976b42ac924f29bc197e23f317edb4, d811603a56444215834731cbff681ca1; no voters:
I20260809 02:10:08.839556 12740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.836133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:45209 (local address 127.5.9.130:48575)
0809 02:10:08.836221 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:08.836233 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:08.836272 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:08.836536 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:08.836537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:08.836539 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:08.836602 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.836604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.837317 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.837318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.838293 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:08.838296 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.839311 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.839312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:08.839389 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.839394 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:10:08.839442 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 02:10:08.839498 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 02:10:08.839708 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.836347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:48575 (local address 127.5.9.129:45209)
0809 02:10:08.836388 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.836390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.836405 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:10:08.836428 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.836428 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.836438 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.836517 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:08.836701 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.837212 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.838404 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.839204 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.839543 (+ 339us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.839560 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.839581 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.839633 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:10:08.839824 12917 raft_consensus.cc:2781] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:10:08.839905 12917 raft_consensus.cc:482] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:10:08.839944 12917 raft_consensus.cc:3037] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:08.840183 12536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec68375b89c9419790da1e6dddebfd7f" candidate_uuid: "b2976b42ac924f29bc197e23f317edb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3955f153a3fb49cfb79a720b1ae457b5" is_pre_election: true
I20260809 02:10:08.840344 12536 raft_consensus.cc:2454] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2976b42ac924f29bc197e23f317edb4 in term 0.
I20260809 02:10:08.840884 12917 raft_consensus.cc:504] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:08.841055 12917 leader_election.cc:288] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [CANDIDATE]: Term 1 election: Requested vote from peers d811603a56444215834731cbff681ca1 (127.5.9.131:35771), 3955f153a3fb49cfb79a720b1ae457b5 (127.5.9.129:45209)
I20260809 02:10:08.841248 12828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec68375b89c9419790da1e6dddebfd7f" candidate_uuid: "b2976b42ac924f29bc197e23f317edb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811603a56444215834731cbff681ca1"
I20260809 02:10:08.841351 12828 raft_consensus.cc:3037] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:08.842527 12828 raft_consensus.cc:2454] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2976b42ac924f29bc197e23f317edb4 in term 1.
I20260809 02:10:08.842722 12536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec68375b89c9419790da1e6dddebfd7f" candidate_uuid: "b2976b42ac924f29bc197e23f317edb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3955f153a3fb49cfb79a720b1ae457b5"
I20260809 02:10:08.842805 12612 leader_election.cc:302] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2976b42ac924f29bc197e23f317edb4, d811603a56444215834731cbff681ca1; no voters:
I20260809 02:10:08.842826 12536 raft_consensus.cc:3037] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:08.843688 12536 raft_consensus.cc:2454] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2976b42ac924f29bc197e23f317edb4 in term 1.
I20260809 02:10:08.843827 12917 raft_consensus.cc:2781] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:08.844058 12917 raft_consensus.cc:686] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [term 1 LEADER]: Becoming Leader. State: Replica: b2976b42ac924f29bc197e23f317edb4, State: Running, Role: LEADER
I20260809 02:10:08.844179 12917 consensus_queue.cc:237] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:08.845237 11552 catalog_manager.cc:5495] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2976b42ac924f29bc197e23f317edb4 (127.5.9.130). New cstate: current_term: 1 leader_uuid: "b2976b42ac924f29bc197e23f317edb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } health_report { overall_health: HEALTHY } } }
I20260809 02:10:08.859261 12909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.856514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58602 (local address 127.5.9.130:46733)
0809 02:10:08.856570 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.856572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.856582 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:10:08.856654 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.856655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.856674 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.856739 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:10:08.856864 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.857381 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.858309 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.858531 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.858605 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.858812 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:10:08.858926 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:10:08.858993 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.859078 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.859100 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 02:10:08.861380 12536 raft_consensus.cc:1261] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5 [term 1 FOLLOWER]: Refusing update from remote peer b2976b42ac924f29bc197e23f317edb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:08.861380 12828 raft_consensus.cc:1261] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1 [term 1 FOLLOWER]: Refusing update from remote peer b2976b42ac924f29bc197e23f317edb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:08.861774 12917 consensus_queue.cc:1035] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:08.861982 12923 consensus_queue.cc:1035] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:08.865698 12926 mvcc.cc:205] Tried to move back new op lower bound from 7316444810693124096 to 7316444810626707456. Current Snapshot: MvccSnapshot[applied={T|T < 7316444810693124096}]
I20260809 02:10:08.866111 12928 mvcc.cc:205] Tried to move back new op lower bound from 7316444810693124096 to 7316444810626707456. Current Snapshot: MvccSnapshot[applied={T|T < 7316444810693124096}]
I20260809 02:10:08.868080 12927 mvcc.cc:205] Tried to move back new op lower bound from 7316444810693124096 to 7316444810626707456. Current Snapshot: MvccSnapshot[applied={T|T < 7316444810693124096}]
I20260809 02:10:08.869580 11552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:08.869742 11552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:08.871258 11552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86d06056e8354223aff7fe1f4a8cc361: Sending DeleteTablet for 3 replicas of tablet ec68375b89c9419790da1e6dddebfd7f
I20260809 02:10:08.871694 12515 tablet_service.cc:1514] Processing DeleteTablet for tablet ec68375b89c9419790da1e6dddebfd7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:08 UTC) from {username='oryx', principal='oryx/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:59550
I20260809 02:10:08.871708 12662 tablet_service.cc:1514] Processing DeleteTablet for tablet ec68375b89c9419790da1e6dddebfd7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:08 UTC) from {username='oryx', principal='oryx/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:58590
I20260809 02:10:08.872085 12807 tablet_service.cc:1514] Processing DeleteTablet for tablet ec68375b89c9419790da1e6dddebfd7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:08 UTC) from {username='oryx', principal='oryx/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:50896
I20260809 02:10:08.872381 12939 tablet_replica.cc:330] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1: stopping tablet replica
I20260809 02:10:08.872514 12939 raft_consensus.cc:2229] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:08.872648 12939 raft_consensus.cc:2258] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:08.872674 12936 tablet_replica.cc:330] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4: stopping tablet replica
I20260809 02:10:08.872768 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 11520
I20260809 02:10:08.874095 12936 raft_consensus.cc:2229] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:10:08.874330 12936 raft_consensus.cc:2258] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:08.874779 12939 ts_tablet_manager.cc:1905] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:08.875006 12936 ts_tablet_manager.cc:1905] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:08.876875 12939 ts_tablet_manager.cc:1918] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:08.877031 12939 log.cc:1196] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/ec68375b89c9419790da1e6dddebfd7f
I20260809 02:10:08.877381 12939 ts_tablet_manager.cc:1939] T ec68375b89c9419790da1e6dddebfd7f P d811603a56444215834731cbff681ca1: Deleting consensus metadata
I20260809 02:10:08.877920 12936 ts_tablet_manager.cc:1918] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:08.878119 12936 log.cc:1196] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/ec68375b89c9419790da1e6dddebfd7f
I20260809 02:10:08.878458 12936 ts_tablet_manager.cc:1939] T ec68375b89c9419790da1e6dddebfd7f P b2976b42ac924f29bc197e23f317edb4: Deleting consensus metadata
W20260809 02:10:08.881927 12609 connection.cc:535] server connection from 127.0.0.1:58590 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 02:10:08.882088 12937 tablet_replica.cc:330] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5: stopping tablet replica
W20260809 02:10:08.882232 12760 connection.cc:535] server connection from 127.0.0.1:50896 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 02:10:08.882285 12937 raft_consensus.cc:2229] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:08.882491 12937 raft_consensus.cc:2258] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:08.883217 12937 ts_tablet_manager.cc:1905] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:08.885380 12937 ts_tablet_manager.cc:1918] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:08.885496 12937 log.cc:1196] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-0/wal/wals/ec68375b89c9419790da1e6dddebfd7f
I20260809 02:10:08.885865 12937 ts_tablet_manager.cc:1939] T ec68375b89c9419790da1e6dddebfd7f P 3955f153a3fb49cfb79a720b1ae457b5: Deleting consensus metadata
W20260809 02:10:08.886408 12468 connection.cc:441] server connection from 127.0.0.1:59550 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59550 (request call id 1) could send its response
I20260809 02:10:08.892416 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.872129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49358 (local address 127.5.9.188:45949)
0809 02:10:08.872177 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.872179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.882122 (+ 9943us) server_negotiation.cc:407] Connection header received
0809 02:10:08.882165 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.882167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.882183 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.882258 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.883518 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.884386 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.885666 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.886036 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.886122 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.888661 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.888662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.888663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.888672 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.890576 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.891047 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.891048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.891049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.891122 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.891339 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.891340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.891341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.891493 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.891599 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.892202 (+ 603us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.892293 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:10:08.900151 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.890687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59052 (local address 127.5.9.189:40825)
0809 02:10:08.890766 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.890768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.890778 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:10:08.890808 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.890809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.890822 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.890881 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.891813 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.892716 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.894029 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.894327 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.894365 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.896439 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.896441 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.896441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.896449 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.898395 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.899050 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.899052 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.899052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.899121 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.899427 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.899428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.899428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.899569 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.899665 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.899908 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.900011 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:10:08.900183 12895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:08.890065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49368 (local address 127.5.9.188:45949)
0809 02:10:08.890775 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:08.890778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:08.890786 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:10:08.890806 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:08.890807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:08.890822 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:08.890881 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:08.891814 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.892685 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.894160 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:08.894532 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:08.894616 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:08.896412 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:08.896414 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:08.896414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:08.896422 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:08.898593 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.899091 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.899092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.899093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.899185 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:08.899432 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:08.899433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:08.899433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:08.899589 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:08.899697 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:08.899929 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 02:10:08.900073 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686}
I20260809 02:10:09.867173 12947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:09.866854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.130:42609)
0809 02:10:09.867030 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:09.867119 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
I20260809 02:10:09.867410 12949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:09.867121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.129:35485)
0809 02:10:09.867259 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:09.867354 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":60,"threads_started":1}
I20260809 02:10:09.867461 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:09.867121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.131:34831)
0809 02:10:09.867258 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:09.867414 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":63,"threads_started":1}
W20260809 02:10:09.867678 12878 heartbeater.cc:644] Failed to heartbeat to 127.5.9.190:39773 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
W20260809 02:10:09.867679 12585 heartbeater.cc:644] Failed to heartbeat to 127.5.9.190:39773 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
W20260809 02:10:09.868124 12735 heartbeater.cc:644] Failed to heartbeat to 127.5.9.190:39773 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
I20260809 02:10:10.371940 12950 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 86d06056e8354223aff7fe1f4a8cc361)
I20260809 02:10:10.372080 12950 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } }
I20260809 02:10:10.372442 12950 leader_election.cc:288] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 86d06056e8354223aff7fe1f4a8cc361 (127.5.9.190:39773), 6cdde3d6ae9041d99deeb13f382cabd6 (127.5.9.188:45949)
I20260809 02:10:10.372437 12951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 86d06056e8354223aff7fe1f4a8cc361)
I20260809 02:10:10.372622 12951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } }
I20260809 02:10:10.372903 12951 leader_election.cc:288] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 86d06056e8354223aff7fe1f4a8cc361 (127.5.9.190:39773), 5fc6594b92bd463ca3ad7d723123c664 (127.5.9.189:40825)
I20260809 02:10:10.373121 12953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.372596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.0.0.1:50292)
0809 02:10:10.372756 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.372824 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":78,"threads_started":1}
I20260809 02:10:10.373178 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.372979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.0.0.1:50302)
0809 02:10:10.373098 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.373142 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":83,"threads_started":1}
W20260809 02:10:10.373250 12380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
W20260809 02:10:10.373248 12177 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
I20260809 02:10:10.373728 12953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.373637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.0.0.1:50312)
0809 02:10:10.373678 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.373703 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 02:10:10.373850 12956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.373719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.0.0.1:50314)
0809 02:10:10.373767 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.373806 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 02:10:10.373816 12177 leader_election.cc:334] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 86d06056e8354223aff7fe1f4a8cc361 (127.5.9.190:39773): Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
W20260809 02:10:10.373915 12380 leader_election.cc:334] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 86d06056e8354223aff7fe1f4a8cc361 (127.5.9.190:39773): Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
I20260809 02:10:10.376443 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.372655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:45949 (local address 127.0.0.1:49374)
0809 02:10:10.372827 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.372840 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:10.372901 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:10.373384 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:10.373385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:10.373394 (+ 9us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:10.373449 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:10.373451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.374413 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.374414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:10.375406 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:10.375408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.376166 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.376167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:10.376234 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:10.376239 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:10:10.376318 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:10:10.376364 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":57,"threads_started":1}
I20260809 02:10:10.376461 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.372775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49374 (local address 127.5.9.188:45949)
0809 02:10:10.373040 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:10.373042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:10.373051 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:10:10.373080 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:10.373081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:10.373093 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:10.373512 (+ 419us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:10.373570 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.374339 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.375493 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.376060 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.376122 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:10.376161 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:10.376328 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 02:10:10.376372 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":46,"threads_started":1}
I20260809 02:10:10.377015 12406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fc6594b92bd463ca3ad7d723123c664" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" is_pre_election: true
I20260809 02:10:10.377149 12406 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fc6594b92bd463ca3ad7d723123c664 in term 1.
I20260809 02:10:10.377192 12957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.373177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59058 (local address 127.5.9.189:40825)
0809 02:10:10.373658 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:10.373660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:10.373669 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:10:10.373691 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:10.373692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:10.373702 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:10.374073 (+ 371us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:10.374225 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.374919 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.376156 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.376689 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.376827 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:10.376846 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:10.377060 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 02:10:10.377126 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":51,"threads_started":1}
I20260809 02:10:10.377252 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.373039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:40825 (local address 127.0.0.1:59058)
0809 02:10:10.373447 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.373460 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:10.373523 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:10.374099 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:10.374100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:10.374104 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:10.374138 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:10.374140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.375004 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.375005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:10.375978 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:10.375980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.376834 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.376835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:10.376923 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:10.376927 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:10:10.377005 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 02:10:10.377051 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":60,"threads_started":1}
I20260809 02:10:10.377349 12175 leader_election.cc:302] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5fc6594b92bd463ca3ad7d723123c664, 6cdde3d6ae9041d99deeb13f382cabd6; no voters: 86d06056e8354223aff7fe1f4a8cc361
I20260809 02:10:10.377460 12950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 02:10:10.377509 12950 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 86d06056e8354223aff7fe1f4a8cc361)
I20260809 02:10:10.377559 12950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:10:10.377569 12202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5fc6594b92bd463ca3ad7d723123c664" is_pre_election: true
I20260809 02:10:10.378357 12950 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } }
I20260809 02:10:10.378616 12202 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6cdde3d6ae9041d99deeb13f382cabd6 in current term 2: Already voted for candidate 5fc6594b92bd463ca3ad7d723123c664 in this term.
I20260809 02:10:10.378630 12950 leader_election.cc:288] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [CANDIDATE]: Term 2 election: Requested vote from peers 86d06056e8354223aff7fe1f4a8cc361 (127.5.9.190:39773), 6cdde3d6ae9041d99deeb13f382cabd6 (127.5.9.188:45949)
I20260809 02:10:10.378789 12406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fc6594b92bd463ca3ad7d723123c664" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6cdde3d6ae9041d99deeb13f382cabd6"
I20260809 02:10:10.378877 12406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:10:10.378983 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.378747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.0.0.1:50318)
0809 02:10:10.378843 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.378947 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 02:10:10.379279 12378 leader_election.cc:302] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cdde3d6ae9041d99deeb13f382cabd6; no voters: 5fc6594b92bd463ca3ad7d723123c664, 86d06056e8354223aff7fe1f4a8cc361
I20260809 02:10:10.379312 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.379223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.0.0.1:50334)
0809 02:10:10.379260 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.379285 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 02:10:10.379374 12177 leader_election.cc:334] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 86d06056e8354223aff7fe1f4a8cc361 (127.5.9.190:39773): Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
I20260809 02:10:10.379783 12406 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fc6594b92bd463ca3ad7d723123c664 in term 2.
I20260809 02:10:10.380008 12175 leader_election.cc:302] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5fc6594b92bd463ca3ad7d723123c664, 6cdde3d6ae9041d99deeb13f382cabd6; no voters: 86d06056e8354223aff7fe1f4a8cc361
I20260809 02:10:10.380095 12950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:10:10.380153 12951 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:10:10.380247 12950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [term 2 LEADER]: Becoming Leader. State: Replica: 5fc6594b92bd463ca3ad7d723123c664, State: Running, Role: LEADER
I20260809 02:10:10.380321 12950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } }
I20260809 02:10:10.380685 12960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fc6594b92bd463ca3ad7d723123c664. Latest consensus state: current_term: 2 leader_uuid: "5fc6594b92bd463ca3ad7d723123c664" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } } }
I20260809 02:10:10.380810 12960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:10.380996 12962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:10:10.381546 12962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:10:10.381676 12962 catalog_manager.cc:1259] Loaded cluster ID: 8c80a661add343e39c95dbfcedfb468e
I20260809 02:10:10.381726 12962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:10:10.381929 12962 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:10:10.381999 12962 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: Loaded TSK: 0
I20260809 02:10:10.382166 12962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:10:10.446758 12189 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:10:10.448660 12189 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:10:10.449934 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.449762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.0.0.1:50364)
0809 02:10:10.449862 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.449901 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 02:10:10.449970 12406 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [term 2 FOLLOWER]: Refusing update from remote peer 5fc6594b92bd463ca3ad7d723123c664: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 02:10:10.450215 12960 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 02:10:10.450388 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.450277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.0.0.1:50370)
0809 02:10:10.450332 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.450361 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 02:10:10.450464 12177 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 -> Peer 86d06056e8354223aff7fe1f4a8cc361 (127.5.9.190:39773): Couldn't send request to peer 86d06056e8354223aff7fe1f4a8cc361. Status: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 02:10:10.451047 12951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fc6594b92bd463ca3ad7d723123c664. Latest consensus state: current_term: 2 leader_uuid: "5fc6594b92bd463ca3ad7d723123c664" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } } }
I20260809 02:10:10.451164 12951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:10:10.451581 12960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "5fc6594b92bd463ca3ad7d723123c664" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } } }
I20260809 02:10:10.451686 12960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:10.451768 12965 mvcc.cc:205] Tried to move back new op lower bound from 7316444817200328704 to 7316444816918302720. Current Snapshot: MvccSnapshot[applied={T|T < 7316444817200328704}]
I20260809 02:10:10.451975 12951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "5fc6594b92bd463ca3ad7d723123c664" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "86d06056e8354223aff7fe1f4a8cc361" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39773 } } peers { permanent_uuid: "5fc6594b92bd463ca3ad7d723123c664" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 40825 } attrs { promote: false } } peers { permanent_uuid: "6cdde3d6ae9041d99deeb13f382cabd6" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 45949 } attrs { promote: false } } }
I20260809 02:10:10.452080 12951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cdde3d6ae9041d99deeb13f382cabd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:10:10.456619 12971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.453417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:46733 (local address 127.0.0.1:58604)
0809 02:10:10.453793 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.453803 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:10.453852 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:10.454044 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:10.454045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:10.454047 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:10.454076 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:10.454077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.454943 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.454944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:10.455730 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:10.455732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.456430 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.456431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:10.456479 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:10.456483 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:10:10.456525 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 02:10:10.456559 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":80,"threads_started":1}
I20260809 02:10:10.456655 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.453667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58604 (local address 127.5.9.130:46733)
0809 02:10:10.453784 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:10.453786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:10.453901 (+ 115us) server_negotiation.cc:407] Connection header received
0809 02:10:10.453929 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:10.453930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:10.453941 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:10.453998 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:10.454144 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.454858 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.455821 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.456359 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.456433 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:10.456447 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:10.456531 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 02:10:10.456568 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":67,"threads_started":1}
I20260809 02:10:10.457078 12953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.453373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:35771 (local address 127.0.0.1:50912)
0809 02:10:10.453541 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.453553 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:10.453646 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:10.453852 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:10.453853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:10.453856 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:10.453919 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:10.453921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.454888 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.454889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:10.455968 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:10.455971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.456802 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.456803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:10.456884 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:10.456886 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:10:10.456958 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:10:10.457012 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":27}
I20260809 02:10:10.457094 12662 tablet_service.cc:1467] Processing CreateTablet for tablet c1d09e76e3f347458e5950ed9ef67441 (DEFAULT_TABLE table=test-table [id=e8eba31ae79d4aaea21ccfd7dc1bb556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:10.457232 12662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1d09e76e3f347458e5950ed9ef67441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:10.457362 12972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.453518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50912 (local address 127.5.9.131:35771)
0809 02:10:10.453689 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:10.453691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:10.453700 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:10:10.453728 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:10.453729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:10.453741 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:10.453843 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:10.454007 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.454808 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.456062 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.456638 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.456713 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:10.456730 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:10.457041 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 02:10:10.457086 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
I20260809 02:10:10.457590 12807 tablet_service.cc:1467] Processing CreateTablet for tablet c1d09e76e3f347458e5950ed9ef67441 (DEFAULT_TABLE table=test-table [id=e8eba31ae79d4aaea21ccfd7dc1bb556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:10.457751 12807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1d09e76e3f347458e5950ed9ef67441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:10.457813 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.453373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:45209 (local address 127.0.0.1:59556)
0809 02:10:10.453443 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.453454 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:10.453497 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:10.454105 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:10.454106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:10.454107 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:10.454151 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:10.454152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.454820 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.454821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:10.455972 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:10.455974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.457589 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.457590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:10.457673 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:10.457678 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:10:10.457710 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 02:10:10.457754 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 02:10:10.458211 12973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.453581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59556 (local address 127.5.9.129:45209)
0809 02:10:10.453993 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:10.453995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:10.454003 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:10:10.454029 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:10.454030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:10.454041 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:10.454103 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:10.454228 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.454733 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.456777 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.457365 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.458022 (+ 657us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:10.458040 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:10.458062 (+ 22us) server_negotiation.cc:299] Negotiation successful
0809 02:10:10.458112 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":136,"threads_started":1}
I20260809 02:10:10.458470 12515 tablet_service.cc:1467] Processing CreateTablet for tablet c1d09e76e3f347458e5950ed9ef67441 (DEFAULT_TABLE table=test-table [id=e8eba31ae79d4aaea21ccfd7dc1bb556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:10.458604 12515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1d09e76e3f347458e5950ed9ef67441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:10.458940 12975 tablet_bootstrap.cc:492] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4: Bootstrap starting.
I20260809 02:10:10.459465 12976 tablet_bootstrap.cc:492] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1: Bootstrap starting.
I20260809 02:10:10.459729 12975 tablet_bootstrap.cc:654] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:10.459944 12976 tablet_bootstrap.cc:654] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:10.460139 12978 tablet_bootstrap.cc:492] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5: Bootstrap starting.
I20260809 02:10:10.460569 12978 tablet_bootstrap.cc:654] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:10.460655 12976 tablet_bootstrap.cc:492] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1: No bootstrap required, opened a new log
I20260809 02:10:10.460716 12976 ts_tablet_manager.cc:1397] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:10.460850 12976 raft_consensus.cc:348] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:10.460933 12976 raft_consensus.cc:374] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:10.460966 12976 raft_consensus.cc:729] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d811603a56444215834731cbff681ca1, State: Initialized, Role: FOLLOWER
I20260809 02:10:10.461027 12976 consensus_queue.cc:260] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:10.461213 12976 ts_tablet_manager.cc:1428] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:10:10.461611 12978 tablet_bootstrap.cc:492] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5: No bootstrap required, opened a new log
I20260809 02:10:10.461692 12978 ts_tablet_manager.cc:1397] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:10:10.461800 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.461430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.131:52771)
0809 02:10:10.461594 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.461660 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260809 02:10:10.461844 12978 raft_consensus.cc:348] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:10.461936 12978 raft_consensus.cc:374] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:10.461987 12978 raft_consensus.cc:729] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3955f153a3fb49cfb79a720b1ae457b5, State: Initialized, Role: FOLLOWER
I20260809 02:10:10.462060 12978 consensus_queue.cc:260] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:10.462143 12975 tablet_bootstrap.cc:492] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4: No bootstrap required, opened a new log
I20260809 02:10:10.462203 12975 ts_tablet_manager.cc:1397] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 02:10:10.462211 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.462125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.131:39905)
0809 02:10:10.462162 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.462187 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 02:10:10.462227 12978 ts_tablet_manager.cc:1428] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 02:10:10.462365 12975 raft_consensus.cc:348] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:10.462451 12975 raft_consensus.cc:374] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:10.462476 12975 raft_consensus.cc:729] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2976b42ac924f29bc197e23f317edb4, State: Initialized, Role: FOLLOWER
I20260809 02:10:10.462520 12975 consensus_queue.cc:260] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:10.462683 12975 ts_tablet_manager.cc:1428] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:10:10.462811 12984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.462486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.129:33213)
0809 02:10:10.462631 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.462675 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":94,"threads_started":1}
W20260809 02:10:10.462869 12587 tablet.cc:2369] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:10:10.463172 12736 heartbeater.cc:503] Master 127.5.9.189:40825 was elected leader, sending a full tablet report...
I20260809 02:10:10.463435 12586 heartbeater.cc:503] Master 127.5.9.189:40825 was elected leader, sending a full tablet report...
I20260809 02:10:10.463502 12984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.463360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.129:55507)
0809 02:10:10.463398 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.463422 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 02:10:10.463539 12879 heartbeater.cc:503] Master 127.5.9.189:40825 was elected leader, sending a full tablet report...
I20260809 02:10:10.464331 12986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.462915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.130:45901)
0809 02:10:10.464164 (+ 1249us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.464278 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1224,"thread_start_us":75,"threads_started":1}
W20260809 02:10:10.530200 12880 tablet.cc:2369] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:10:10.631739 12737 tablet.cc:2369] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:10:10.726500 12985 raft_consensus.cc:482] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:10:10.726624 12985 raft_consensus.cc:504] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:10.726933 12985 leader_election.cc:288] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d811603a56444215834731cbff681ca1 (127.5.9.131:35771), 3955f153a3fb49cfb79a720b1ae457b5 (127.5.9.129:45209)
I20260809 02:10:10.727105 12536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1d09e76e3f347458e5950ed9ef67441" candidate_uuid: "b2976b42ac924f29bc197e23f317edb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3955f153a3fb49cfb79a720b1ae457b5" is_pre_election: true
I20260809 02:10:10.727105 12828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1d09e76e3f347458e5950ed9ef67441" candidate_uuid: "b2976b42ac924f29bc197e23f317edb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811603a56444215834731cbff681ca1" is_pre_election: true
I20260809 02:10:10.727212 12536 raft_consensus.cc:2454] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2976b42ac924f29bc197e23f317edb4 in term 0.
I20260809 02:10:10.727212 12828 raft_consensus.cc:2454] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2976b42ac924f29bc197e23f317edb4 in term 0.
I20260809 02:10:10.727378 12610 leader_election.cc:302] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3955f153a3fb49cfb79a720b1ae457b5, b2976b42ac924f29bc197e23f317edb4; no voters:
I20260809 02:10:10.727519 12985 raft_consensus.cc:2781] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:10:10.727566 12985 raft_consensus.cc:482] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:10:10.727581 12985 raft_consensus.cc:3037] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:10.728333 12985 raft_consensus.cc:504] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:10.728504 12985 leader_election.cc:288] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [CANDIDATE]: Term 1 election: Requested vote from peers d811603a56444215834731cbff681ca1 (127.5.9.131:35771), 3955f153a3fb49cfb79a720b1ae457b5 (127.5.9.129:45209)
I20260809 02:10:10.728665 12828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1d09e76e3f347458e5950ed9ef67441" candidate_uuid: "b2976b42ac924f29bc197e23f317edb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d811603a56444215834731cbff681ca1"
I20260809 02:10:10.728755 12828 raft_consensus.cc:3037] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:10.728746 12536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1d09e76e3f347458e5950ed9ef67441" candidate_uuid: "b2976b42ac924f29bc197e23f317edb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3955f153a3fb49cfb79a720b1ae457b5"
I20260809 02:10:10.728833 12536 raft_consensus.cc:3037] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:10.729597 12828 raft_consensus.cc:2454] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2976b42ac924f29bc197e23f317edb4 in term 1.
I20260809 02:10:10.729597 12536 raft_consensus.cc:2454] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2976b42ac924f29bc197e23f317edb4 in term 1.
I20260809 02:10:10.729843 12610 leader_election.cc:302] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3955f153a3fb49cfb79a720b1ae457b5, b2976b42ac924f29bc197e23f317edb4; no voters:
I20260809 02:10:10.730012 12985 raft_consensus.cc:2781] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:10.730207 12985 raft_consensus.cc:686] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [term 1 LEADER]: Becoming Leader. State: Replica: b2976b42ac924f29bc197e23f317edb4, State: Running, Role: LEADER
I20260809 02:10:10.730285 12985 consensus_queue.cc:237] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } }
I20260809 02:10:10.730607 12986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.730481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.130:39429)
0809 02:10:10.730515 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.730558 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 02:10:10.731215 12189 catalog_manager.cc:5495] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2976b42ac924f29bc197e23f317edb4 (127.5.9.130). New cstate: current_term: 1 leader_uuid: "b2976b42ac924f29bc197e23f317edb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2976b42ac924f29bc197e23f317edb4" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 46733 } health_report { overall_health: HEALTHY } } }
I20260809 02:10:10.744601 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.741460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58608 (local address 127.5.9.130:46733)
0809 02:10:10.741511 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:10.741513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:10.741523 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:10:10.741601 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:10.741602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:10.741655 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:10.741714 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:10:10.741892 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.742381 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.743492 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:10.743901 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:10.743972 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:10.744192 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:10:10.744234 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:10:10.744279 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:10.744465 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 02:10:10.744494 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:10:10.746673 12536 raft_consensus.cc:1261] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5 [term 1 FOLLOWER]: Refusing update from remote peer b2976b42ac924f29bc197e23f317edb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:10.746794 12828 raft_consensus.cc:1261] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1 [term 1 FOLLOWER]: Refusing update from remote peer b2976b42ac924f29bc197e23f317edb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:10.747139 12985 consensus_queue.cc:1035] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3955f153a3fb49cfb79a720b1ae457b5" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:10.747278 12987 consensus_queue.cc:1035] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d811603a56444215834731cbff681ca1" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:10.748045 12984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.747840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.129:40141)
0809 02:10:10.747958 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.748000 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260809 02:10:10.748353 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.748221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.131:46387)
0809 02:10:10.748280 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.748313 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 02:10:10.750660 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.750569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.131:56525)
0809 02:10:10.750602 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.750628 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 02:10:10.751001 12986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.749584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.130:44989)
0809 02:10:10.750926 (+ 1342us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.750959 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1323}
I20260809 02:10:10.751098 12189 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:10.751194 12189 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:10.751401 12986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.751191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.130:59037)
0809 02:10:10.751343 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.751370 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":108,"threads_started":1}
I20260809 02:10:10.752697 12984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.752590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:39773 (local address 127.5.9.129:41199)
0809 02:10:10.752628 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.752664 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 02:10:10.754180 12189 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5fc6594b92bd463ca3ad7d723123c664: Sending DeleteTablet for 3 replicas of tablet c1d09e76e3f347458e5950ed9ef67441
I20260809 02:10:10.754541 12807 tablet_service.cc:1514] Processing DeleteTablet for tablet c1d09e76e3f347458e5950ed9ef67441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:10 UTC) from {username='oryx', principal='oryx/127.5.9.189@KRBTEST.COM'} at 127.0.0.1:50912
I20260809 02:10:10.754608 12515 tablet_service.cc:1514] Processing DeleteTablet for tablet c1d09e76e3f347458e5950ed9ef67441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:10 UTC) from {username='oryx', principal='oryx/127.5.9.189@KRBTEST.COM'} at 127.0.0.1:59556
I20260809 02:10:10.754833 12662 tablet_service.cc:1514] Processing DeleteTablet for tablet c1d09e76e3f347458e5950ed9ef67441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:10 UTC) from {username='oryx', principal='oryx/127.5.9.189@KRBTEST.COM'} at 127.0.0.1:58604
I20260809 02:10:10.754861 13003 tablet_replica.cc:330] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5: stopping tablet replica
I20260809 02:10:10.754917 13003 raft_consensus.cc:2229] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:10.754967 13003 raft_consensus.cc:2258] T c1d09e76e3f347458e5950ed9ef67441 P 3955f153a3fb49cfb79a720b1ae457b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:10.755136 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 12355
I20260809 02:10:10.756392 13004 tablet_replica.cc:330] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4: stopping tablet replica
I20260809 02:10:10.756455 13004 raft_consensus.cc:2229] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:10:10.756516 13004 raft_consensus.cc:2258] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:10.756942 13004 ts_tablet_manager.cc:1905] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:10.757997 13002 tablet_replica.cc:330] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1: stopping tablet replica
I20260809 02:10:10.758069 13002 raft_consensus.cc:2229] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:10.758124 13002 raft_consensus.cc:2258] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:10.758594 13002 ts_tablet_manager.cc:1905] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:10.759163 13004 ts_tablet_manager.cc:1918] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:10.759220 13004 log.cc:1196] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/c1d09e76e3f347458e5950ed9ef67441
I20260809 02:10:10.759497 13004 ts_tablet_manager.cc:1939] T c1d09e76e3f347458e5950ed9ef67441 P b2976b42ac924f29bc197e23f317edb4: Deleting consensus metadata
I20260809 02:10:10.760392 13002 ts_tablet_manager.cc:1918] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:10.760453 13002 log.cc:1196] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/c1d09e76e3f347458e5950ed9ef67441
I20260809 02:10:10.760718 13002 ts_tablet_manager.cc:1939] T c1d09e76e3f347458e5950ed9ef67441 P d811603a56444215834731cbff681ca1: Deleting consensus metadata
I20260809 02:10:10.761096 12176 catalog_manager.cc:4841] TS d811603a56444215834731cbff681ca1 (127.5.9.131:35771): tablet c1d09e76e3f347458e5950ed9ef67441 (table test-table [id=e8eba31ae79d4aaea21ccfd7dc1bb556]) successfully deleted
I20260809 02:10:10.761621 12177 catalog_manager.cc:4841] TS b2976b42ac924f29bc197e23f317edb4 (127.5.9.130:46733): tablet c1d09e76e3f347458e5950ed9ef67441 (table test-table [id=e8eba31ae79d4aaea21ccfd7dc1bb556]) successfully deleted
I20260809 02:10:10.766011 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:10.765822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:45209 (local address 127.0.0.1:59562)
0809 02:10:10.765925 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:10.765960 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:45209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 02:10:10.766011 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 12592
W20260809 02:10:10.767529 12175 catalog_manager.cc:4584] TS 3955f153a3fb49cfb79a720b1ae457b5 (127.5.9.129:45209): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c1d09e76e3f347458e5950ed9ef67441: Network error: Client connection negotiation failed: client connection to 127.5.9.129:45209: connect: Connection refused (error 111)
I20260809 02:10:10.774298 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 12741
I20260809 02:10:10.782202 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 12050
I20260809 02:10:10.788877 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 12252
2026-08-09T02:10:10Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5061 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:10:10 dist-test-slave-p3t5 krb5kdc[13007](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:10:10 dist-test-slave-p3t5 krb5kdc[13007](info): set up 2 sockets
Aug 09 02:10:10 dist-test-slave-p3t5 krb5kdc[13007](info): commencing operation
krb5kdc: starting...
W20260809 02:10:11.898504 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:10:11 dist-test-slave-p3t5 krb5kdc[13007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:10:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:10:12Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.190.keytab.
Entry for principal oryx/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.190.keytab.
I20260809 02:10:12.042440 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:43827
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.190.keytab
--principal=oryx/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:43827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:12.131364 13021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:12.131605 13021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:12.131630 13021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:12.132905 13021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:12.132961 13021 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 02:10:12.132975 13021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:12.132989 13021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:12.133002 13021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:12.133056 13021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:10:12.134783 13021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:43827
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.190.keytab
--principal=oryx/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:43827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:12.135082 13021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:12.135381 13021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:12.140695 13029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:12.140722 13027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:12.140695 13026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:12.141045 13021 server_base.cc:1034] running on GCE node
I20260809 02:10:12.141322 13021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:12.141706 13021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:12.142894 13021 hybrid_clock.cc:648] HybridClock initialized: now 1786241412142865 us; error 56 us; skew 500 ppm
Aug 09 02:10:12 dist-test-slave-p3t5 krb5kdc[13007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241412, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:12.145936 13021 init.cc:377] Logged in from keytab as oryx/127.5.9.190@KRBTEST.COM (short username oryx)
I20260809 02:10:12.146454 13021 webserver.cc:460] Webserver started at http://127.5.9.190:33635/ using document root <none> and password file <none>
I20260809 02:10:12.146757 13021 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:12.146811 13021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:12.146960 13021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:12.148145 13021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "63b785ddfae2467ebf214a4ab87139db"
format_stamp: "Formatted at 2026-08-09 02:10:12 on dist-test-slave-p3t5"
server_key: "0b14cb71e2af113165310239af8335e7"
server_key_iv: "4946fb38e33cb735415d215dc476546a"
server_key_version: "encryptionkey@0"
I20260809 02:10:12.148562 13021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "63b785ddfae2467ebf214a4ab87139db"
format_stamp: "Formatted at 2026-08-09 02:10:12 on dist-test-slave-p3t5"
server_key: "0b14cb71e2af113165310239af8335e7"
server_key_iv: "4946fb38e33cb735415d215dc476546a"
server_key_version: "encryptionkey@0"
I20260809 02:10:12.150134 13021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:10:12.151014 13039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:12.151244 13021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:12.151355 13021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "63b785ddfae2467ebf214a4ab87139db"
format_stamp: "Formatted at 2026-08-09 02:10:12 on dist-test-slave-p3t5"
server_key: "0b14cb71e2af113165310239af8335e7"
server_key_iv: "4946fb38e33cb735415d215dc476546a"
server_key_version: "encryptionkey@0"
I20260809 02:10:12.151438 13021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:12.165174 13021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:12.167856 13021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:12.168031 13021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:12.173234 13021 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:43827
I20260809 02:10:12.173350 13108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:43827 every 8 connection(s)
I20260809 02:10:12.173818 13021 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:10:12.174671 13109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:12.177179 13109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db: Bootstrap starting.
I20260809 02:10:12.177990 13109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:12.178339 13109 log.cc:824] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:12.179261 13109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db: No bootstrap required, opened a new log
I20260809 02:10:12.180366 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 13021
I20260809 02:10:12.180469 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:10:12.180665 5158 external_mini_cluster.cc:1428] Setting key 213ee15bc8853b1b4f1b281385a91fcd
I20260809 02:10:12.181679 13109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b785ddfae2467ebf214a4ab87139db" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43827 } }
I20260809 02:10:12.181864 13109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:12.181911 13109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63b785ddfae2467ebf214a4ab87139db, State: Initialized, Role: FOLLOWER
I20260809 02:10:12.182088 13109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b785ddfae2467ebf214a4ab87139db" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43827 } }
I20260809 02:10:12.182189 13109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:10:12.182235 13109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:10:12.182286 13109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:12.182897 13109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b785ddfae2467ebf214a4ab87139db" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43827 } }
I20260809 02:10:12.183027 13109 leader_election.cc:302] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63b785ddfae2467ebf214a4ab87139db; no voters:
I20260809 02:10:12.183259 13109 leader_election.cc:288] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:10:12.183295 13114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:12.183404 13114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [term 1 LEADER]: Becoming Leader. State: Replica: 63b785ddfae2467ebf214a4ab87139db, State: Running, Role: LEADER
I20260809 02:10:12.183492 13114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b785ddfae2467ebf214a4ab87139db" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43827 } }
I20260809 02:10:12.183586 13109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:12.183846 13114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63b785ddfae2467ebf214a4ab87139db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b785ddfae2467ebf214a4ab87139db" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43827 } } }
I20260809 02:10:12.183940 13114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:12.184077 13114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63b785ddfae2467ebf214a4ab87139db. Latest consensus state: current_term: 1 leader_uuid: "63b785ddfae2467ebf214a4ab87139db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b785ddfae2467ebf214a4ab87139db" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43827 } } }
I20260809 02:10:12.184154 13114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db [sys.catalog]: This master's current role is: LEADER
W20260809 02:10:12.185484 13133 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:10:12.185546 13133 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:10:12.185597 13119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:10:12.185791 13119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:10:12.187529 13119 catalog_manager.cc:1347] Generated new cluster ID: d79ee1d18b5e4db898b7482f51ee8f84
I20260809 02:10:12.187592 13119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:10:12 dist-test-slave-p3t5 krb5kdc[13007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM
I20260809 02:10:12.193161 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.181487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38238 (local address 127.5.9.190:43827)
0809 02:10:12.181690 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:12.181700 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:12.181736 (+ 36us) server_negotiation.cc:407] Connection header received
0809 02:10:12.181860 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:12.181866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:12.181949 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:12.182103 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:12.182785 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.184197 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.185297 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.185987 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.186106 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.189191 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:12.189195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:12.189195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:12.189209 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:12.191275 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:12.191749 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:12.191752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:12.191753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:12.191891 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:12.192130 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:12.192131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:12.192131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:12.192477 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:12.192634 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:12.192781 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 02:10:12.192929 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
I20260809 02:10:12.208384 13119 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:10:12.209012 13119 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:10:12.211629 13119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db: Generated new TSK 0
I20260809 02:10:12.211879 13119 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.129.keytab.
Entry for principal oryx/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.129.keytab.
I20260809 02:10:12.245941 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:43827
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.129.keytab
--principal=oryx/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:12.337255 13140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:12.337482 13140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:12.337509 13140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:12.338826 13140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:12.338889 13140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:12.338913 13140 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:10:12.338933 13140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:10:12.340583 13140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.129.keytab
--principal=oryx/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.9.190:43827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:12.340870 13140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:12.341130 13140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:12.346803 13146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:12.346803 13148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:12.346834 13145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:12.347047 13140 server_base.cc:1034] running on GCE node
I20260809 02:10:12.347282 13140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:12.347568 13140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:12.348776 13140 hybrid_clock.cc:648] HybridClock initialized: now 1786241412348756 us; error 35 us; skew 500 ppm
Aug 09 02:10:12 dist-test-slave-p3t5 krb5kdc[13007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241412, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:12.351768 13140 init.cc:377] Logged in from keytab as oryx/127.5.9.129@KRBTEST.COM (short username oryx)
I20260809 02:10:12.352298 13140 webserver.cc:460] Webserver started at http://127.5.9.129:45193/ using document root <none> and password file <none>
I20260809 02:10:12.352541 13140 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:12.352612 13140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:12.352766 13140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:12.353998 13140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "bd321365f16e4e2bb0573bb361226c8e"
format_stamp: "Formatted at 2026-08-09 02:10:12 on dist-test-slave-p3t5"
server_key: "ef3eae96bc19e84141cf2caabbf6df0d"
server_key_iv: "c78cd5b2735b879915d789e9e0989a44"
server_key_version: "encryptionkey@0"
I20260809 02:10:12.354386 13140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "bd321365f16e4e2bb0573bb361226c8e"
format_stamp: "Formatted at 2026-08-09 02:10:12 on dist-test-slave-p3t5"
server_key: "ef3eae96bc19e84141cf2caabbf6df0d"
server_key_iv: "c78cd5b2735b879915d789e9e0989a44"
server_key_version: "encryptionkey@0"
I20260809 02:10:12.355981 13140 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:10:12.356843 13156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:12.357048 13140 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:12.357163 13140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "bd321365f16e4e2bb0573bb361226c8e"
format_stamp: "Formatted at 2026-08-09 02:10:12 on dist-test-slave-p3t5"
server_key: "ef3eae96bc19e84141cf2caabbf6df0d"
server_key_iv: "c78cd5b2735b879915d789e9e0989a44"
server_key_version: "encryptionkey@0"
I20260809 02:10:12.357244 13140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:12.388131 13140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:12.390714 13140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:12.390923 13140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:12.391511 13140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:12.391582 13140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:12.391633 13140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:12.391662 13140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:10:12 dist-test-slave-p3t5 krb5kdc[13007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241412, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM
I20260809 02:10:12.399670 13140 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:43681
I20260809 02:10:12.400043 13283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:43681 every 8 connection(s)
I20260809 02:10:12.400151 13140 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:10:12.402810 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.392133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:46203 (local address 127.5.9.190:43827)
0809 02:10:12.392187 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:12.392190 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:12.392436 (+ 246us) server_negotiation.cc:407] Connection header received
0809 02:10:12.393142 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:12.393144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:12.393165 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:12.393242 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:12.394364 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.395253 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.396422 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.396670 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.396759 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.399678 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:12.399680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:12.399680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:12.399692 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:12.401332 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:12.401898 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:12.401899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:12.401900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:12.401977 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:12.402208 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:12.402209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:12.402209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:12.402326 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:12.402374 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:12.402659 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 02:10:12.402742 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 02:10:12.402894 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.392114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43827 (local address 127.5.9.129:46203)
0809 02:10:12.392302 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:12.392346 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:12.392972 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:12.393361 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:12.393365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:12.393842 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:12.394238 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.394246 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.395397 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.395398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.396307 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.396311 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.396791 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.396792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.396937 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.397593 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:12.397603 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:12.399555 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:12.401440 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:12.401447 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:12.401449 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:12.401810 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:12.402049 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:12.402050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:12.402050 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:12.402161 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:12.402365 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:12.402368 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:12.402505 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 02:10:12.402581 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:10:12.402722 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":116,"threads_started":1}
I20260809 02:10:12.403304 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 13140
I20260809 02:10:12.403427 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:10:12.403530 13284 heartbeater.cc:347] Connected to a master server at 127.5.9.190:43827
I20260809 02:10:12.403614 13284 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:12.403638 5158 external_mini_cluster.cc:1428] Setting key c51484bc9633c26b6be5068091dcf527
I20260809 02:10:12.403811 13284 heartbeater.cc:511] Master 127.5.9.190:43827 requested a full tablet report, sending...
I20260809 02:10:12.404497 13060 ts_manager.cc:194] Registered new tserver with Master: bd321365f16e4e2bb0573bb361226c8e (127.5.9.129:43681)
I20260809 02:10:12.405457 13060 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:46203
I20260809 02:10:12.412012 13291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.404671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43827 (local address 127.5.9.129:40795)
0809 02:10:12.404832 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:12.404845 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:12.404900 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:12.405043 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:12.405043 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:12.405250 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:12.405325 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.405327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.406311 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.406312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.406850 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.406852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.407383 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.407384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.407459 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.408042 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:12.408048 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:12.408427 (+ 379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:12.410659 (+ 2232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:12.410660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:12.410661 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:12.410862 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:12.411167 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:12.411168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:12.411169 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:12.411302 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:12.411695 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:12.411696 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:12.411746 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 02:10:12.411806 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 02:10:12.411922 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260809 02:10:12.412091 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.404859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:40795 (local address 127.5.9.190:43827)
0809 02:10:12.404924 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:12.404926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:12.404937 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:10:12.404961 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:12.404962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:12.404973 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:12.405033 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:12.405411 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.406189 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.406948 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.407270 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.407355 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.408559 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:12.408560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:12.408561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:12.408569 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:12.410524 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:12.410979 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:12.410981 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:12.410982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:12.411057 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:12.411447 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:12.411449 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:12.411449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:12.411615 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:12.411698 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:12.411864 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 02:10:12.411986 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
WARNING: no policy specified for oryx/127.5.9.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.130.keytab.
Entry for principal oryx/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.130.keytab.
I20260809 02:10:12.437038 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:43827
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.130.keytab
--principal=oryx/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:12.525848 13294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:12.526087 13294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:12.526113 13294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:12.527839 13294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:12.527899 13294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:12.527926 13294 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:10:12.527945 13294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:10:12.529500 13294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.130.keytab
--principal=oryx/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.9.190:43827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:12.529793 13294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:12.530009 13294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:12.535236 13301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:12.535236 13303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:12.535387 13294 server_base.cc:1034] running on GCE node
W20260809 02:10:12.535300 13300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:12.535805 13294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:12.536096 13294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:12.537271 13294 hybrid_clock.cc:648] HybridClock initialized: now 1786241412537247 us; error 40 us; skew 500 ppm
Aug 09 02:10:12 dist-test-slave-p3t5 krb5kdc[13007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241412, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:12.540169 13294 init.cc:377] Logged in from keytab as oryx/127.5.9.130@KRBTEST.COM (short username oryx)
I20260809 02:10:12.540716 13294 webserver.cc:460] Webserver started at http://127.5.9.130:44673/ using document root <none> and password file <none>
I20260809 02:10:12.540980 13294 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:12.541052 13294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:12.541209 13294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:12.542421 13294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "e490ec8957904bd8b266229bee365a87"
format_stamp: "Formatted at 2026-08-09 02:10:12 on dist-test-slave-p3t5"
server_key: "08e221b5c751d26d6e56190e9915076d"
server_key_iv: "ad10c7b8db268c9ad1d120dabb1ee7fc"
server_key_version: "encryptionkey@0"
I20260809 02:10:12.542822 13294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "e490ec8957904bd8b266229bee365a87"
format_stamp: "Formatted at 2026-08-09 02:10:12 on dist-test-slave-p3t5"
server_key: "08e221b5c751d26d6e56190e9915076d"
server_key_iv: "ad10c7b8db268c9ad1d120dabb1ee7fc"
server_key_version: "encryptionkey@0"
I20260809 02:10:12.544308 13294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:10:12.545156 13310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:12.545361 13294 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:12.545477 13294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "e490ec8957904bd8b266229bee365a87"
format_stamp: "Formatted at 2026-08-09 02:10:12 on dist-test-slave-p3t5"
server_key: "08e221b5c751d26d6e56190e9915076d"
server_key_iv: "ad10c7b8db268c9ad1d120dabb1ee7fc"
server_key_version: "encryptionkey@0"
I20260809 02:10:12.545555 13294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:12.560590 13294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:12.564090 13294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:12.564301 13294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:12.565093 13294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:12.565156 13294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:12.565198 13294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:12.565220 13294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:12.572863 13294 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:40137
I20260809 02:10:12.572980 13433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:40137 every 8 connection(s)
I20260809 02:10:12.573338 13294 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:10:12.573585 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 13294
I20260809 02:10:12.573729 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:10:12.573966 5158 external_mini_cluster.cc:1428] Setting key 22c80b9fed7bf847447c3324b33f2d47
Aug 09 02:10:12 dist-test-slave-p3t5 krb5kdc[13007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241412, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM
I20260809 02:10:12.578155 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.565518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:53631 (local address 127.5.9.190:43827)
0809 02:10:12.565592 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:12.565595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:12.565997 (+ 402us) server_negotiation.cc:407] Connection header received
0809 02:10:12.567637 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:12.567638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:12.567655 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:12.567731 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:12.568828 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.569749 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.571501 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.571835 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.571922 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.574942 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:12.574944 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:12.574944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:12.574953 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:12.576503 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:12.576991 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:12.576993 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:12.576993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:12.577066 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:12.577344 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:12.577345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:12.577346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:12.577483 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:12.577560 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:12.577998 (+ 438us) server_negotiation.cc:299] Negotiation successful
0809 02:10:12.578082 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:10:12.578476 13320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.565514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43827 (local address 127.5.9.130:53631)
0809 02:10:12.565829 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:12.565884 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:12.566558 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:12.567903 (+ 1345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:12.567909 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:12.568379 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:12.568728 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.568737 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.569865 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.569866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.571389 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.571394 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.571953 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.571954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.572133 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.572949 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:12.572960 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:12.574834 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:12.576626 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:12.576635 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:12.576638 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:12.576896 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:12.577141 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:12.577142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:12.577143 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:12.577270 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:12.577562 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:12.577571 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:12.577834 (+ 263us) client_negotiation.cc:769] Sending connection context
0809 02:10:12.577962 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 02:10:12.578163 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":108,"threads_started":1}
I20260809 02:10:12.579048 13434 heartbeater.cc:347] Connected to a master server at 127.5.9.190:43827
I20260809 02:10:12.579154 13434 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:12.579387 13434 heartbeater.cc:511] Master 127.5.9.190:43827 requested a full tablet report, sending...
I20260809 02:10:12.579829 13059 ts_manager.cc:194] Registered new tserver with Master: e490ec8957904bd8b266229bee365a87 (127.5.9.130:40137)
I20260809 02:10:12.580271 13059 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:53631
WARNING: no policy specified for oryx/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.9.131@KRBTEST.COM" created.
I20260809 02:10:12.587060 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.580674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:36437 (local address 127.5.9.190:43827)
0809 02:10:12.580723 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:12.580725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:12.580879 (+ 154us) server_negotiation.cc:407] Connection header received
0809 02:10:12.580976 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:12.580977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:12.580989 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:12.581018 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:12.581462 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.582033 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.582766 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.583039 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.583114 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.584215 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:12.584217 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:12.584218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:12.584225 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:12.585772 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:12.586148 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:12.586149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:12.586149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:12.586214 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:12.586482 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:12.586483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:12.586483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:12.586649 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:12.586710 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:12.586902 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 02:10:12.586990 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:10:12.587190 13441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.580594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43827 (local address 127.5.9.130:36437)
0809 02:10:12.580808 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:12.580818 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:12.580905 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:12.581081 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:12.581082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:12.581304 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:12.581363 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.581365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.582132 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.582133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.582665 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.582667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.583114 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.583115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.583186 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.583712 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:12.583717 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:12.584113 (+ 396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:12.585860 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:12.585861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:12.585862 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:12.586055 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:12.586305 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:12.586306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:12.586306 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:12.586412 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:12.586716 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:12.586717 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:12.586770 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 02:10:12.586877 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 02:10:12.587030 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":79,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.131.keytab.
Entry for principal oryx/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.131.keytab.
I20260809 02:10:12.605698 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:43827
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.131.keytab
--principal=oryx/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:12.692095 13443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:12.692317 13443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:12.692343 13443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:12.693499 13443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:12.693534 13443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:12.693554 13443 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:10:12.693571 13443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:10:12.695256 13443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.131.keytab
--principal=oryx/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.9.190:43827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:12.695467 13443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:12.695662 13443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:12.700858 13449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:12.700907 13448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:12.701104 13443 server_base.cc:1034] running on GCE node
W20260809 02:10:12.701121 13452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:12.701403 13443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:12.701733 13443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:12.702915 13443 hybrid_clock.cc:648] HybridClock initialized: now 1786241412702887 us; error 40 us; skew 500 ppm
Aug 09 02:10:12 dist-test-slave-p3t5 krb5kdc[13007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241412, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:12.705709 13443 init.cc:377] Logged in from keytab as oryx/127.5.9.131@KRBTEST.COM (short username oryx)
I20260809 02:10:12.706225 13443 webserver.cc:460] Webserver started at http://127.5.9.131:36255/ using document root <none> and password file <none>
I20260809 02:10:12.706496 13443 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:12.706570 13443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:12.706736 13443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:12.707880 13443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "fa59c2c778014739b88a05f5808528fb"
format_stamp: "Formatted at 2026-08-09 02:10:12 on dist-test-slave-p3t5"
server_key: "4e38cc95913b217d205ae05282862507"
server_key_iv: "dc3338c9f11871443ff0ed7bca879d7a"
server_key_version: "encryptionkey@0"
I20260809 02:10:12.708276 13443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "fa59c2c778014739b88a05f5808528fb"
format_stamp: "Formatted at 2026-08-09 02:10:12 on dist-test-slave-p3t5"
server_key: "4e38cc95913b217d205ae05282862507"
server_key_iv: "dc3338c9f11871443ff0ed7bca879d7a"
server_key_version: "encryptionkey@0"
I20260809 02:10:12.709777 13443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:10:12.710680 13459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:12.710879 13443 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:12.711014 13443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "fa59c2c778014739b88a05f5808528fb"
format_stamp: "Formatted at 2026-08-09 02:10:12 on dist-test-slave-p3t5"
server_key: "4e38cc95913b217d205ae05282862507"
server_key_iv: "dc3338c9f11871443ff0ed7bca879d7a"
server_key_version: "encryptionkey@0"
I20260809 02:10:12.711102 13443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:12.718640 13443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:12.721154 13443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:12.721333 13443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:12.722039 13443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:12.722101 13443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:12.722146 13443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:12.722168 13443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:10:12 dist-test-slave-p3t5 krb5kdc[13007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241412, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM
I20260809 02:10:12.729981 13443 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:43045
I20260809 02:10:12.730082 13580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:43045 every 8 connection(s)
I20260809 02:10:12.730458 13443 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:10:12.731974 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 13443
I20260809 02:10:12.732074 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:10:12.732275 5158 external_mini_cluster.cc:1428] Setting key 6412e6bfbb110b570a70ca78a8ac0f2d
I20260809 02:10:12.733583 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.722492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:54265 (local address 127.5.9.190:43827)
0809 02:10:12.722567 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:12.722570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:12.722835 (+ 265us) server_negotiation.cc:407] Connection header received
0809 02:10:12.723614 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:12.723616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:12.723630 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:12.723699 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:12.724664 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.725189 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.727000 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.727241 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.727341 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.730149 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:12.730151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:12.730151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:12.730161 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:12.731976 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:12.732439 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:12.732441 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:12.732441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:12.732524 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:12.732859 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:12.732860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:12.732860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:12.732998 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:12.733080 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:12.733381 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 02:10:12.733493 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:10:12.733786 13468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.722463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43827 (local address 127.5.9.131:54265)
0809 02:10:12.722659 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:12.722714 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:12.723480 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:12.723782 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:12.723788 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:12.724223 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:12.724549 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.724557 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.725269 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.725271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.726869 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.726873 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.727376 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.727377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.727548 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.728142 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:12.728155 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:12.730033 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:12.732071 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:12.732078 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:12.732080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:12.732321 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:12.732605 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:12.732607 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:12.732607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:12.732784 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:12.733082 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:12.733090 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:12.733237 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 02:10:12.733344 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 02:10:12.733519 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":110,"threads_started":1}
I20260809 02:10:12.734283 13581 heartbeater.cc:347] Connected to a master server at 127.5.9.190:43827
I20260809 02:10:12.734372 13581 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:12.734565 13581 heartbeater.cc:511] Master 127.5.9.190:43827 requested a full tablet report, sending...
I20260809 02:10:12.735028 13060 ts_manager.cc:194] Registered new tserver with Master: fa59c2c778014739b88a05f5808528fb (127.5.9.131:43045)
I20260809 02:10:12.735488 13060 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:54265
I20260809 02:10:12.741253 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.735511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:60931 (local address 127.5.9.190:43827)
0809 02:10:12.735557 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:12.735559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:12.735795 (+ 236us) server_negotiation.cc:407] Connection header received
0809 02:10:12.735815 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:12.735816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:12.735824 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:12.735893 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:12.736261 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.736753 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.737476 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.737755 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.737834 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.738742 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:12.738744 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:12.738744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:12.738752 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:12.740060 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:12.740419 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:12.740420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:12.740421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:12.740482 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:12.740707 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:12.740708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:12.740708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:12.740829 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:12.740908 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:12.741074 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 02:10:12.741152 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:10:12.741269 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.735416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43827 (local address 127.5.9.131:60931)
0809 02:10:12.735574 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:12.735587 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:12.735653 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:12.735899 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:12.735900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:12.736111 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:12.736169 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.736171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.736851 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.736852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.737363 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.737364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.737832 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.737833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.737905 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.738337 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:12.738341 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:12.738647 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:12.740161 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:12.740162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:12.740163 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:12.740343 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:12.740561 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:12.740562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:12.740562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:12.740631 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:12.740899 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:12.740899 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:12.740946 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 02:10:12.741037 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 02:10:12.741154 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260809 02:10:12.744199 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:10:12 dist-test-slave-p3t5 krb5kdc[13007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241412, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:10:12 dist-test-slave-p3t5 krb5kdc[13007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:12 dist-test-slave-p3t5 krb5kdc[13007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 02:10:12.768919 13113 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:12.762859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38254 (local address 127.5.9.190:43827)
0809 02:10:12.762940 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:12.762942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:12.762951 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:10:12.762988 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:12.762988 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:12.762998 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:12.763066 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:12.763684 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.764203 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.764965 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.765166 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.765258 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.768788 (+ 3530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38254: BlockingRecv error: recv got EOF from 127.0.0.1:38254 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 02:10:12 dist-test-slave-p3t5 krb5kdc[13007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241412, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM
I20260809 02:10:12.778062 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.769791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38264 (local address 127.5.9.190:43827)
0809 02:10:12.769876 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:12.769879 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:12.769992 (+ 113us) server_negotiation.cc:407] Connection header received
0809 02:10:12.770077 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:12.770078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:12.770089 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:12.770146 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:12.770659 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.771164 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.771901 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.772112 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.772195 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.774643 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:12.774645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:12.774645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:12.774654 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:12.776640 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:12.777138 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:12.777139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:12.777140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:12.777213 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:12.777430 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:12.777431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:12.777432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:12.777570 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:12.777740 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:12.777854 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 02:10:12.777949 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:10:12.779850 13060 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:10:12.781519 13060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:10:12.788970 13605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.784193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:40137 (local address 127.0.0.1:45372)
0809 02:10:12.785332 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:12.785354 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:12.785406 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:12.785605 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:12.785606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:12.785608 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:12.785674 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.785676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.786506 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.786507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.787848 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.787851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.788727 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.788727 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.788796 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.788799 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:10:12.788867 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 02:10:12.788903 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1116,"mutex_wait_us":7,"thread_start_us":131,"threads_started":1}
I20260809 02:10:12.789026 13607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.784479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45372 (local address 127.5.9.130:40137)
0809 02:10:12.784705 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:12.784710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:12.785452 (+ 742us) server_negotiation.cc:407] Connection header received
0809 02:10:12.785496 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:12.785498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:12.785512 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:12.785607 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:12.785735 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.786411 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.787943 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.788655 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.788728 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.788797 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:12.788884 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 02:10:12.788921 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":109,"threads_started":1}
I20260809 02:10:12.789131 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.784710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59432 (local address 127.5.9.129:43681)
0809 02:10:12.784922 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:12.784926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:12.784935 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:10:12.784974 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:12.784976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:12.785017 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:12.785102 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:12.785372 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.786619 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.787962 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.788478 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.788574 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.788662 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:12.788971 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 02:10:12.789022 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":109,"threads_started":1}
I20260809 02:10:12.789258 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.784480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39838 (local address 127.5.9.131:43045)
0809 02:10:12.784738 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:12.784742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:12.785082 (+ 340us) server_negotiation.cc:407] Connection header received
0809 02:10:12.785129 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:12.785132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:12.785149 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:12.785240 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:12.785437 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.786636 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.787779 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.788535 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.788609 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.788685 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:12.789012 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 02:10:12.789065 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":89,"threads_started":1}
I20260809 02:10:12.788925 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.784189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:43681 (local address 127.0.0.1:59432)
0809 02:10:12.784422 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:12.784471 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:12.784550 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:12.785164 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:12.785199 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:12.785202 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:12.785283 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.785286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.786709 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.786710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.787856 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.787858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.788575 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.788576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.788675 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.788691 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 02:10:12.788766 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 02:10:12.788830 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":58,"threads_started":1}
I20260809 02:10:12.789700 13606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.784189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:43045 (local address 127.0.0.1:39838)
0809 02:10:12.784956 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:12.784967 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:12.785022 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:12.785252 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:12.785255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:12.785261 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:12.785310 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.785312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.786750 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.786751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.787682 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.787685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.788725 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.788726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.788806 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.788810 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:10:12.788863 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 02:10:12.788917 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":5,"spinlock_wait_cycles":896,"thread_start_us":123,"threads_started":1}
I20260809 02:10:12.790450 13363 tablet_service.cc:1467] Processing CreateTablet for tablet 368f70d8964f446a8a37d56798c69b5f (DEFAULT_TABLE table=test-table [id=17fd852e06a34dbeabc2dd1da8715406]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:12.790450 13513 tablet_service.cc:1467] Processing CreateTablet for tablet 368f70d8964f446a8a37d56798c69b5f (DEFAULT_TABLE table=test-table [id=17fd852e06a34dbeabc2dd1da8715406]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:12.790833 13513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 368f70d8964f446a8a37d56798c69b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:12.790849 13363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 368f70d8964f446a8a37d56798c69b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:12.794165 13610 tablet_bootstrap.cc:492] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb: Bootstrap starting.
I20260809 02:10:12.794764 13611 tablet_bootstrap.cc:492] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87: Bootstrap starting.
I20260809 02:10:12.794914 13610 tablet_bootstrap.cc:654] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:12.794677 13213 tablet_service.cc:1467] Processing CreateTablet for tablet 368f70d8964f446a8a37d56798c69b5f (DEFAULT_TABLE table=test-table [id=17fd852e06a34dbeabc2dd1da8715406]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:12.795212 13213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 368f70d8964f446a8a37d56798c69b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:12.795289 13610 log.cc:824] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:12.795469 13611 tablet_bootstrap.cc:654] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:12.795903 13611 log.cc:824] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:12.796200 13610 tablet_bootstrap.cc:492] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb: No bootstrap required, opened a new log
I20260809 02:10:12.796299 13610 ts_tablet_manager.cc:1397] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:10:12.796765 13611 tablet_bootstrap.cc:492] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87: No bootstrap required, opened a new log
I20260809 02:10:12.796937 13611 ts_tablet_manager.cc:1397] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:10:12.798764 13614 tablet_bootstrap.cc:492] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e: Bootstrap starting.
I20260809 02:10:12.798686 13610 raft_consensus.cc:348] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } }
I20260809 02:10:12.798877 13610 raft_consensus.cc:374] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:12.798928 13610 raft_consensus.cc:729] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa59c2c778014739b88a05f5808528fb, State: Initialized, Role: FOLLOWER
I20260809 02:10:12.799053 13610 consensus_queue.cc:260] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } }
I20260809 02:10:12.799387 13610 ts_tablet_manager.cc:1428] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:10:12.799444 13614 tablet_bootstrap.cc:654] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:12.799397 13581 heartbeater.cc:503] Master 127.5.9.190:43827 was elected leader, sending a full tablet report...
I20260809 02:10:12.799557 13611 raft_consensus.cc:348] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } }
I20260809 02:10:12.799691 13611 raft_consensus.cc:374] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:12.799764 13611 raft_consensus.cc:729] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e490ec8957904bd8b266229bee365a87, State: Initialized, Role: FOLLOWER
I20260809 02:10:12.799782 13614 log.cc:824] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:12.799913 13611 consensus_queue.cc:260] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } }
I20260809 02:10:12.800194 13611 ts_tablet_manager.cc:1428] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 02:10:12.800362 13434 heartbeater.cc:503] Master 127.5.9.190:43827 was elected leader, sending a full tablet report...
I20260809 02:10:12.801056 13614 tablet_bootstrap.cc:492] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e: No bootstrap required, opened a new log
I20260809 02:10:12.801155 13614 ts_tablet_manager.cc:1397] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:10:12.802733 13614 raft_consensus.cc:348] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } }
I20260809 02:10:12.802887 13614 raft_consensus.cc:374] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:12.802912 13614 raft_consensus.cc:729] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd321365f16e4e2bb0573bb361226c8e, State: Initialized, Role: FOLLOWER
I20260809 02:10:12.803076 13614 consensus_queue.cc:260] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } }
I20260809 02:10:12.803380 13614 ts_tablet_manager.cc:1428] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 02:10:12.803491 13284 heartbeater.cc:503] Master 127.5.9.190:43827 was elected leader, sending a full tablet report...
W20260809 02:10:12.824076 13435 tablet.cc:2369] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:10:12.879650 13615 raft_consensus.cc:482] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:10:12.879813 13615 raft_consensus.cc:504] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } }
I20260809 02:10:12.880267 13615 leader_election.cc:288] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd321365f16e4e2bb0573bb361226c8e (127.5.9.129:43681), e490ec8957904bd8b266229bee365a87 (127.5.9.130:40137)
I20260809 02:10:12.883445 13607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.880575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:46643 (local address 127.5.9.130:40137)
0809 02:10:12.880626 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:12.880628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:12.880646 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:10:12.880691 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:12.880692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:12.880702 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:12.880755 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:12.880958 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.881596 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.882635 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.883036 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.883125 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.883138 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:12.883307 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 02:10:12.883350 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:10:12.883457 13468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.880491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:40137 (local address 127.5.9.131:46643)
0809 02:10:12.880566 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:12.880581 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:12.880633 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:12.880786 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:12.880787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:12.880789 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:12.880868 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.880870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.881727 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.881728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.882535 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.882537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.883135 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.883136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.883232 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.883235 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:10:12.883301 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 02:10:12.883345 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":25}
I20260809 02:10:12.883471 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.880546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:37053 (local address 127.5.9.129:43681)
0809 02:10:12.880596 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:12.880598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:12.881024 (+ 426us) server_negotiation.cc:407] Connection header received
0809 02:10:12.881071 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:12.881072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:12.881083 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:12.881123 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:12.881299 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.881767 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.882698 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.883086 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.883172 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.883182 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:12.883348 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 02:10:12.883399 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:10:12.883603 13619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.880497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:43681 (local address 127.5.9.131:37053)
0809 02:10:12.880942 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:12.880954 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:12.881018 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:12.881152 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:12.881153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:12.881155 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:12.881235 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.881237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.881869 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.881870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.882601 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:12.882603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.883176 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.883177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:12.883258 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.883261 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:10:12.883340 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:10:12.883380 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":190,"thread_start_us":141,"threads_started":1}
I20260809 02:10:12.883950 13385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "368f70d8964f446a8a37d56798c69b5f" candidate_uuid: "fa59c2c778014739b88a05f5808528fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e490ec8957904bd8b266229bee365a87" is_pre_election: true
I20260809 02:10:12.884152 13385 raft_consensus.cc:2454] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa59c2c778014739b88a05f5808528fb in term 0.
I20260809 02:10:12.884136 13234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "368f70d8964f446a8a37d56798c69b5f" candidate_uuid: "fa59c2c778014739b88a05f5808528fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd321365f16e4e2bb0573bb361226c8e" is_pre_election: true
I20260809 02:10:12.884316 13234 raft_consensus.cc:2454] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa59c2c778014739b88a05f5808528fb in term 0.
I20260809 02:10:12.884398 13462 leader_election.cc:302] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e490ec8957904bd8b266229bee365a87, fa59c2c778014739b88a05f5808528fb; no voters:
I20260809 02:10:12.884531 13615 raft_consensus.cc:2781] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:10:12.884593 13615 raft_consensus.cc:482] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:10:12.884649 13615 raft_consensus.cc:3037] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:12.885588 13615 raft_consensus.cc:504] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } }
I20260809 02:10:12.885767 13615 leader_election.cc:288] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [CANDIDATE]: Term 1 election: Requested vote from peers bd321365f16e4e2bb0573bb361226c8e (127.5.9.129:43681), e490ec8957904bd8b266229bee365a87 (127.5.9.130:40137)
I20260809 02:10:12.885958 13234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "368f70d8964f446a8a37d56798c69b5f" candidate_uuid: "fa59c2c778014739b88a05f5808528fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd321365f16e4e2bb0573bb361226c8e"
I20260809 02:10:12.885967 13385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "368f70d8964f446a8a37d56798c69b5f" candidate_uuid: "fa59c2c778014739b88a05f5808528fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e490ec8957904bd8b266229bee365a87"
I20260809 02:10:12.886051 13234 raft_consensus.cc:3037] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:12.886075 13385 raft_consensus.cc:3037] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:12.886818 13385 raft_consensus.cc:2454] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa59c2c778014739b88a05f5808528fb in term 1.
I20260809 02:10:12.886839 13234 raft_consensus.cc:2454] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa59c2c778014739b88a05f5808528fb in term 1.
I20260809 02:10:12.887038 13462 leader_election.cc:302] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e490ec8957904bd8b266229bee365a87, fa59c2c778014739b88a05f5808528fb; no voters:
I20260809 02:10:12.887133 13615 raft_consensus.cc:2781] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:12.887290 13615 raft_consensus.cc:686] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [term 1 LEADER]: Becoming Leader. State: Replica: fa59c2c778014739b88a05f5808528fb, State: Running, Role: LEADER
I20260809 02:10:12.887409 13615 consensus_queue.cc:237] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } }
I20260809 02:10:12.888321 13059 catalog_manager.cc:5495] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb reported cstate change: term changed from 0 to 1, leader changed from <none> to fa59c2c778014739b88a05f5808528fb (127.5.9.131). New cstate: current_term: 1 leader_uuid: "fa59c2c778014739b88a05f5808528fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } health_report { overall_health: UNKNOWN } } }
W20260809 02:10:12.901172 13285 tablet.cc:2369] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:10:12.905932 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:12.903135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39854 (local address 127.5.9.131:43045)
0809 02:10:12.903201 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:12.903203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:12.903216 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:10:12.903405 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:12.903406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:12.903427 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:12.903498 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:10:12.903654 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.904147 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.905027 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:12.905222 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:12.905297 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:12.905509 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:10:12.905660 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:10:12.905766 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:12.905798 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 02:10:12.905824 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:10:12.908059 13385 raft_consensus.cc:1261] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Refusing update from remote peer fa59c2c778014739b88a05f5808528fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:12.908059 13234 raft_consensus.cc:1261] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Refusing update from remote peer fa59c2c778014739b88a05f5808528fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:12.908449 13615 consensus_queue.cc:1035] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:12.908675 13620 consensus_queue.cc:1035] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:12.911206 13623 mvcc.cc:205] Tried to move back new op lower bound from 7316444827267854336 to 7316444827187453952. Current Snapshot: MvccSnapshot[applied={T|T < 7316444827267854336}]
I20260809 02:10:12.912050 13626 mvcc.cc:205] Tried to move back new op lower bound from 7316444827267854336 to 7316444827187453952. Current Snapshot: MvccSnapshot[applied={T|T < 7316444827267854336}]
I20260809 02:10:12.913802 13059 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:12.913923 13059 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:12.914897 13059 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db: Sending DeleteTablet for 3 replicas of tablet 368f70d8964f446a8a37d56798c69b5f
I20260809 02:10:12.915343 13213 tablet_service.cc:1514] Processing DeleteTablet for tablet 368f70d8964f446a8a37d56798c69b5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:12 UTC) from {username='oryx', principal='oryx/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:59432
I20260809 02:10:12.915318 13513 tablet_service.cc:1514] Processing DeleteTablet for tablet 368f70d8964f446a8a37d56798c69b5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:12 UTC) from {username='oryx', principal='oryx/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:39838
I20260809 02:10:12.915589 13363 tablet_service.cc:1514] Processing DeleteTablet for tablet 368f70d8964f446a8a37d56798c69b5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:12 UTC) from {username='oryx', principal='oryx/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:45372
I20260809 02:10:12.915637 13059 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:10:12.915813 13059 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:10:12.915838 13634 tablet_replica.cc:330] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb: stopping tablet replica
I20260809 02:10:12.915931 13634 raft_consensus.cc:2229] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:10:12.916062 13634 raft_consensus.cc:2258] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:12.916397 13635 tablet_replica.cc:330] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87: stopping tablet replica
I20260809 02:10:12.916559 13635 raft_consensus.cc:2229] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:12.916603 13634 ts_tablet_manager.cc:1905] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:12.916774 13635 raft_consensus.cc:2258] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:12.917305 13635 ts_tablet_manager.cc:1905] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:12.917404 13513 tablet_service.cc:1467] Processing CreateTablet for tablet 847ba6c895504761ab34d5ddeaad33bb (DEFAULT_TABLE table=test-table [id=1b5262b173e64217857e6020c0e39af5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:12.917550 13513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847ba6c895504761ab34d5ddeaad33bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:12.918493 13634 ts_tablet_manager.cc:1918] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:12.918578 13634 log.cc:1196] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/368f70d8964f446a8a37d56798c69b5f
I20260809 02:10:12.918854 13634 ts_tablet_manager.cc:1939] T 368f70d8964f446a8a37d56798c69b5f P fa59c2c778014739b88a05f5808528fb: Deleting consensus metadata
I20260809 02:10:12.919111 13610 tablet_bootstrap.cc:492] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb: Bootstrap starting.
I20260809 02:10:12.919104 13633 tablet_replica.cc:330] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e: stopping tablet replica
I20260809 02:10:12.919384 13633 raft_consensus.cc:2229] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:12.919421 13363 tablet_service.cc:1467] Processing CreateTablet for tablet 847ba6c895504761ab34d5ddeaad33bb (DEFAULT_TABLE table=test-table [id=1b5262b173e64217857e6020c0e39af5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:12.919539 13633 raft_consensus.cc:2258] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:12.919564 13610 tablet_bootstrap.cc:654] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:12.919695 13363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847ba6c895504761ab34d5ddeaad33bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:12.919759 13213 tablet_service.cc:1467] Processing CreateTablet for tablet 847ba6c895504761ab34d5ddeaad33bb (DEFAULT_TABLE table=test-table [id=1b5262b173e64217857e6020c0e39af5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:12.919890 13213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847ba6c895504761ab34d5ddeaad33bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:12.920125 13633 ts_tablet_manager.cc:1905] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:12.920920 13611 tablet_bootstrap.cc:492] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87: Bootstrap starting.
I20260809 02:10:12.921221 13635 ts_tablet_manager.cc:1918] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:12.921314 13614 tablet_bootstrap.cc:492] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e: Bootstrap starting.
I20260809 02:10:12.921375 13611 tablet_bootstrap.cc:654] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:12.921482 13635 log.cc:1196] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/368f70d8964f446a8a37d56798c69b5f
I20260809 02:10:12.921741 13614 tablet_bootstrap.cc:654] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:12.921938 13635 ts_tablet_manager.cc:1939] T 368f70d8964f446a8a37d56798c69b5f P e490ec8957904bd8b266229bee365a87: Deleting consensus metadata
I20260809 02:10:12.922142 13043 catalog_manager.cc:4841] TS fa59c2c778014739b88a05f5808528fb (127.5.9.131:43045): tablet 368f70d8964f446a8a37d56798c69b5f (table test-table [id=17fd852e06a34dbeabc2dd1da8715406]) successfully deleted
I20260809 02:10:12.922194 13611 tablet_bootstrap.cc:492] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87: No bootstrap required, opened a new log
I20260809 02:10:12.922235 13611 ts_tablet_manager.cc:1397] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:10:12.922398 13611 raft_consensus.cc:348] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:12.922480 13611 raft_consensus.cc:374] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:12.922508 13611 raft_consensus.cc:729] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e490ec8957904bd8b266229bee365a87, State: Initialized, Role: FOLLOWER
I20260809 02:10:12.922556 13611 consensus_queue.cc:260] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:12.922585 13042 catalog_manager.cc:4841] TS e490ec8957904bd8b266229bee365a87 (127.5.9.130:40137): tablet 368f70d8964f446a8a37d56798c69b5f (table test-table [id=17fd852e06a34dbeabc2dd1da8715406]) successfully deleted
I20260809 02:10:12.922653 13611 ts_tablet_manager.cc:1428] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 02:10:12.923214 13633 ts_tablet_manager.cc:1918] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:12.923298 13633 log.cc:1196] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/wal/wals/368f70d8964f446a8a37d56798c69b5f
I20260809 02:10:12.923604 13633 ts_tablet_manager.cc:1939] T 368f70d8964f446a8a37d56798c69b5f P bd321365f16e4e2bb0573bb361226c8e: Deleting consensus metadata
I20260809 02:10:12.924045 13041 catalog_manager.cc:4841] TS bd321365f16e4e2bb0573bb361226c8e (127.5.9.129:43681): tablet 368f70d8964f446a8a37d56798c69b5f (table test-table [id=17fd852e06a34dbeabc2dd1da8715406]) successfully deleted
I20260809 02:10:12.926429 13614 tablet_bootstrap.cc:492] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e: No bootstrap required, opened a new log
I20260809 02:10:12.926517 13614 ts_tablet_manager.cc:1397] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260809 02:10:12.926766 13610 tablet_bootstrap.cc:492] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb: No bootstrap required, opened a new log
I20260809 02:10:12.926831 13610 ts_tablet_manager.cc:1397] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260809 02:10:12.926954 13614 raft_consensus.cc:348] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:12.926994 13610 raft_consensus.cc:348] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:12.927059 13614 raft_consensus.cc:374] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:12.927075 13610 raft_consensus.cc:374] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:12.927090 13614 raft_consensus.cc:729] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd321365f16e4e2bb0573bb361226c8e, State: Initialized, Role: FOLLOWER
I20260809 02:10:12.927098 13610 raft_consensus.cc:729] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa59c2c778014739b88a05f5808528fb, State: Initialized, Role: FOLLOWER
I20260809 02:10:12.927136 13614 consensus_queue.cc:260] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:12.927145 13610 consensus_queue.cc:260] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:12.927228 13610 ts_tablet_manager.cc:1428] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:10:12.927265 13614 ts_tablet_manager.cc:1428] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20260809 02:10:12.981146 13582 tablet.cc:2369] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:10:13.064878 13630 raft_consensus.cc:482] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:10:13.065004 13630 raft_consensus.cc:504] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.065199 13630 leader_election.cc:288] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd321365f16e4e2bb0573bb361226c8e (127.5.9.129:43681), e490ec8957904bd8b266229bee365a87 (127.5.9.130:40137)
I20260809 02:10:13.065487 13385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847ba6c895504761ab34d5ddeaad33bb" candidate_uuid: "fa59c2c778014739b88a05f5808528fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e490ec8957904bd8b266229bee365a87" is_pre_election: true
I20260809 02:10:13.065531 13234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847ba6c895504761ab34d5ddeaad33bb" candidate_uuid: "fa59c2c778014739b88a05f5808528fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd321365f16e4e2bb0573bb361226c8e" is_pre_election: true
I20260809 02:10:13.065608 13385 raft_consensus.cc:2454] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa59c2c778014739b88a05f5808528fb in term 0.
I20260809 02:10:13.065712 13234 raft_consensus.cc:2454] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa59c2c778014739b88a05f5808528fb in term 0.
I20260809 02:10:13.065923 13461 leader_election.cc:302] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd321365f16e4e2bb0573bb361226c8e, fa59c2c778014739b88a05f5808528fb; no voters:
I20260809 02:10:13.066045 13630 raft_consensus.cc:2781] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:10:13.066105 13630 raft_consensus.cc:482] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:10:13.066121 13630 raft_consensus.cc:3037] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:13.066883 13630 raft_consensus.cc:504] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.067046 13630 leader_election.cc:288] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [CANDIDATE]: Term 1 election: Requested vote from peers bd321365f16e4e2bb0573bb361226c8e (127.5.9.129:43681), e490ec8957904bd8b266229bee365a87 (127.5.9.130:40137)
I20260809 02:10:13.067210 13385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847ba6c895504761ab34d5ddeaad33bb" candidate_uuid: "fa59c2c778014739b88a05f5808528fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e490ec8957904bd8b266229bee365a87"
I20260809 02:10:13.067234 13234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847ba6c895504761ab34d5ddeaad33bb" candidate_uuid: "fa59c2c778014739b88a05f5808528fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd321365f16e4e2bb0573bb361226c8e"
I20260809 02:10:13.067315 13234 raft_consensus.cc:3037] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:13.067317 13385 raft_consensus.cc:3037] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:13.068030 13385 raft_consensus.cc:2454] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa59c2c778014739b88a05f5808528fb in term 1.
I20260809 02:10:13.068064 13234 raft_consensus.cc:2454] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa59c2c778014739b88a05f5808528fb in term 1.
I20260809 02:10:13.068228 13462 leader_election.cc:302] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e490ec8957904bd8b266229bee365a87, fa59c2c778014739b88a05f5808528fb; no voters:
I20260809 02:10:13.068352 13630 raft_consensus.cc:2781] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:13.068430 13630 raft_consensus.cc:686] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [term 1 LEADER]: Becoming Leader. State: Replica: fa59c2c778014739b88a05f5808528fb, State: Running, Role: LEADER
I20260809 02:10:13.068486 13630 consensus_queue.cc:237] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.069128 13058 catalog_manager.cc:5495] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb reported cstate change: term changed from 0 to 1, leader changed from <none> to fa59c2c778014739b88a05f5808528fb (127.5.9.131). New cstate: current_term: 1 leader_uuid: "fa59c2c778014739b88a05f5808528fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } health_report { overall_health: HEALTHY } } }
Aug 09 02:10:13 dist-test-slave-p3t5 krb5kdc[13007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241412, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.190@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM
I20260809 02:10:13.133997 13646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.126497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43827 (local address 127.0.0.1:38278)
0809 02:10:13.126682 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:13.126692 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:13.126775 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:13.126933 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:13.126934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:13.127254 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:13.127309 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:13.127311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.128028 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.128028 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:13.128587 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:13.128589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.128957 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.128958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:13.129011 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.129678 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:13.129689 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:13.131195 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:13.132837 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:13.132845 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:13.132847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:13.133072 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:13.133259 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:13.133259 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:13.133259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:13.133329 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:13.133578 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:13.133585 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:13.133693 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 02:10:13.133758 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 02:10:13.133884 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":96,"threads_started":1}
I20260809 02:10:13.134055 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.126596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38278 (local address 127.5.9.190:43827)
0809 02:10:13.126716 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:13.126718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:13.126771 (+ 53us) server_negotiation.cc:407] Connection header received
0809 02:10:13.126828 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:13.126829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:13.126838 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:13.126893 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:13.127426 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.127942 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.128680 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.128881 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.128943 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.131281 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:13.131282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:13.131282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:13.131289 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:13.132749 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:13.133141 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:13.133142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:13.133143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:13.133197 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:13.133393 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:13.133393 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:13.133393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:13.133518 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:13.133572 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:13.133782 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 02:10:13.133939 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":83}
I20260809 02:10:13.136200 13058 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:38278:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 02:10:13.136441 13058 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:10:13.137966 13213 tablet_service.cc:1467] Processing CreateTablet for tablet add5832112a94f20b862639dde21cc81 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4f57014148c04b93b70aafeb15858bf3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:10:13.138028 13513 tablet_service.cc:1467] Processing CreateTablet for tablet add5832112a94f20b862639dde21cc81 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4f57014148c04b93b70aafeb15858bf3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:10:13.138170 13213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet add5832112a94f20b862639dde21cc81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:13.138217 13513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet add5832112a94f20b862639dde21cc81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:13.139401 13610 tablet_bootstrap.cc:492] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb: Bootstrap starting.
I20260809 02:10:13.139560 13363 tablet_service.cc:1467] Processing CreateTablet for tablet add5832112a94f20b862639dde21cc81 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4f57014148c04b93b70aafeb15858bf3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:10:13.139707 13363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet add5832112a94f20b862639dde21cc81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:13.139768 13610 tablet_bootstrap.cc:654] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:13.140439 13610 tablet_bootstrap.cc:492] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb: No bootstrap required, opened a new log
I20260809 02:10:13.140486 13610 ts_tablet_manager.cc:1397] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:13.140625 13610 raft_consensus.cc:348] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.140697 13610 raft_consensus.cc:374] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:13.140710 13610 raft_consensus.cc:729] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa59c2c778014739b88a05f5808528fb, State: Initialized, Role: FOLLOWER
I20260809 02:10:13.140764 13610 consensus_queue.cc:260] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.140872 13610 ts_tablet_manager.cc:1428] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:13.140887 13630 tablet_replica.cc:439] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } }
I20260809 02:10:13.140962 13630 tablet_replica.cc:442] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:10:13.141103 13614 tablet_bootstrap.cc:492] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e: Bootstrap starting.
I20260809 02:10:13.141600 13614 tablet_bootstrap.cc:654] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:13.141752 13611 tablet_bootstrap.cc:492] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87: Bootstrap starting.
I20260809 02:10:13.142124 13611 tablet_bootstrap.cc:654] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:13.142612 13614 tablet_bootstrap.cc:492] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e: No bootstrap required, opened a new log
I20260809 02:10:13.142689 13614 ts_tablet_manager.cc:1397] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 02:10:13.142831 13614 raft_consensus.cc:348] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.142887 13614 raft_consensus.cc:374] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:13.142901 13614 raft_consensus.cc:729] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd321365f16e4e2bb0573bb361226c8e, State: Initialized, Role: FOLLOWER
I20260809 02:10:13.142959 13614 consensus_queue.cc:260] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.142990 13611 tablet_bootstrap.cc:492] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87: No bootstrap required, opened a new log
I20260809 02:10:13.143028 13611 ts_tablet_manager.cc:1397] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:10:13.143054 13614 ts_tablet_manager.cc:1428] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:13.143124 13618 tablet_replica.cc:439] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } }
I20260809 02:10:13.143170 13611 raft_consensus.cc:348] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.143222 13618 tablet_replica.cc:442] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:10:13.143241 13611 raft_consensus.cc:374] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:13.143267 13611 raft_consensus.cc:729] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e490ec8957904bd8b266229bee365a87, State: Initialized, Role: FOLLOWER
I20260809 02:10:13.143313 13611 consensus_queue.cc:260] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.143440 13611 ts_tablet_manager.cc:1428] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:13.143512 13616 tablet_replica.cc:439] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } }
I20260809 02:10:13.143631 13616 tablet_replica.cc:442] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:10:13.180266 13630 raft_consensus.cc:482] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:10:13.180375 13630 raft_consensus.cc:504] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.180588 13630 leader_election.cc:288] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd321365f16e4e2bb0573bb361226c8e (127.5.9.129:43681), e490ec8957904bd8b266229bee365a87 (127.5.9.130:40137)
I20260809 02:10:13.180889 13234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "add5832112a94f20b862639dde21cc81" candidate_uuid: "fa59c2c778014739b88a05f5808528fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd321365f16e4e2bb0573bb361226c8e" is_pre_election: true
I20260809 02:10:13.180874 13385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "add5832112a94f20b862639dde21cc81" candidate_uuid: "fa59c2c778014739b88a05f5808528fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e490ec8957904bd8b266229bee365a87" is_pre_election: true
I20260809 02:10:13.180992 13385 raft_consensus.cc:2454] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa59c2c778014739b88a05f5808528fb in term 0.
I20260809 02:10:13.181018 13234 raft_consensus.cc:2454] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa59c2c778014739b88a05f5808528fb in term 0.
I20260809 02:10:13.181185 13462 leader_election.cc:302] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: bd321365f16e4e2bb0573bb361226c8e, e490ec8957904bd8b266229bee365a87, fa59c2c778014739b88a05f5808528fb; no voters:
I20260809 02:10:13.181320 13630 raft_consensus.cc:2781] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:10:13.181401 13630 raft_consensus.cc:482] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:10:13.181432 13630 raft_consensus.cc:3037] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:13.182189 13630 raft_consensus.cc:504] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.182354 13630 leader_election.cc:288] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [CANDIDATE]: Term 1 election: Requested vote from peers bd321365f16e4e2bb0573bb361226c8e (127.5.9.129:43681), e490ec8957904bd8b266229bee365a87 (127.5.9.130:40137)
I20260809 02:10:13.182528 13234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "add5832112a94f20b862639dde21cc81" candidate_uuid: "fa59c2c778014739b88a05f5808528fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd321365f16e4e2bb0573bb361226c8e"
I20260809 02:10:13.182613 13234 raft_consensus.cc:3037] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:13.182567 13385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "add5832112a94f20b862639dde21cc81" candidate_uuid: "fa59c2c778014739b88a05f5808528fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e490ec8957904bd8b266229bee365a87"
I20260809 02:10:13.182685 13385 raft_consensus.cc:3037] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:13.183315 13234 raft_consensus.cc:2454] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa59c2c778014739b88a05f5808528fb in term 1.
I20260809 02:10:13.183398 13385 raft_consensus.cc:2454] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa59c2c778014739b88a05f5808528fb in term 1.
I20260809 02:10:13.183521 13461 leader_election.cc:302] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd321365f16e4e2bb0573bb361226c8e, fa59c2c778014739b88a05f5808528fb; no voters:
I20260809 02:10:13.183648 13630 raft_consensus.cc:2781] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:13.183820 13630 raft_consensus.cc:686] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [term 1 LEADER]: Becoming Leader. State: Replica: fa59c2c778014739b88a05f5808528fb, State: Running, Role: LEADER
I20260809 02:10:13.184012 13630 consensus_queue.cc:237] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.184566 13620 tablet_replica.cc:439] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb: TxnStatusTablet state changed. Reason: New leader fa59c2c778014739b88a05f5808528fb. Latest consensus state: current_term: 1 leader_uuid: "fa59c2c778014739b88a05f5808528fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } }
I20260809 02:10:13.184692 13620 tablet_replica.cc:442] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:10:13.184839 13059 catalog_manager.cc:5495] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb reported cstate change: term changed from 0 to 1, leader changed from <none> to fa59c2c778014739b88a05f5808528fb (127.5.9.131). New cstate: current_term: 1 leader_uuid: "fa59c2c778014739b88a05f5808528fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } health_report { overall_health: HEALTHY } } }
I20260809 02:10:13.185230 13651 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 02:10:13.185385 13651 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 02:10:13.202325 13646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.199447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:43045 (local address 127.0.0.1:39858)
0809 02:10:13.199536 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:13.199549 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:13.199608 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:13.199905 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:13.199906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:13.199907 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:10:13.199969 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:13.199971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.200864 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.200865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:13.201442 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:13.201443 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.201922 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.201923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:13.201977 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.201994 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:10:13.202177 (+ 183us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:10:13.202183 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:10:13.202219 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 02:10:13.202260 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 02:10:13.202339 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.199692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39858 (local address 127.5.9.131:43045)
0809 02:10:13.199766 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:13.199768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:13.199779 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:10:13.199809 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:13.199811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:13.199822 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:13.199885 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:10:13.200038 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.200776 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.201520 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.201844 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.201919 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.202079 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:10:13.202115 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:10:13.202170 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:13.202223 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 02:10:13.202253 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:10:13.203574 13385 raft_consensus.cc:1261] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Refusing update from remote peer fa59c2c778014739b88a05f5808528fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:13.203576 13234 raft_consensus.cc:1261] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Refusing update from remote peer fa59c2c778014739b88a05f5808528fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:13.203817 13620 consensus_queue.cc:1035] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:13.203974 13630 consensus_queue.cc:1035] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:13.204516 13618 tablet_replica.cc:439] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e: TxnStatusTablet state changed. Reason: New leader fa59c2c778014739b88a05f5808528fb. Latest consensus state: current_term: 1 leader_uuid: "fa59c2c778014739b88a05f5808528fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } }
I20260809 02:10:13.204629 13618 tablet_replica.cc:442] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:10:13.204965 13616 tablet_replica.cc:439] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87: TxnStatusTablet state changed. Reason: New leader fa59c2c778014739b88a05f5808528fb. Latest consensus state: current_term: 1 leader_uuid: "fa59c2c778014739b88a05f5808528fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } }
I20260809 02:10:13.205066 13616 tablet_replica.cc:442] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:10:13.205510 13620 tablet_replica.cc:439] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fa59c2c778014739b88a05f5808528fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } }
I20260809 02:10:13.205603 13620 tablet_replica.cc:442] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:10:13.206290 13618 tablet_replica.cc:439] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fa59c2c778014739b88a05f5808528fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } }
I20260809 02:10:13.206394 13618 tablet_replica.cc:442] T add5832112a94f20b862639dde21cc81 P bd321365f16e4e2bb0573bb361226c8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:10:13.207351 13615 tablet_replica.cc:439] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fa59c2c778014739b88a05f5808528fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } }
I20260809 02:10:13.207445 13615 tablet_replica.cc:442] T add5832112a94f20b862639dde21cc81 P fa59c2c778014739b88a05f5808528fb: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:10:13.207792 13616 tablet_replica.cc:439] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fa59c2c778014739b88a05f5808528fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } } }
I20260809 02:10:13.207897 13616 tablet_replica.cc:442] T add5832112a94f20b862639dde21cc81 P e490ec8957904bd8b266229bee365a87: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:10:13.212455 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.209829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:46771 (local address 127.5.9.131:43045)
0809 02:10:13.209902 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:13.209904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:13.209916 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:10:13.209962 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:13.209963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:13.209970 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:13.210014 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:10:13.210133 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.210663 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.211767 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.211985 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.212037 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.212231 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:10:13.212259 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:10:13.212297 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:13.212350 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 02:10:13.212367 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 02:10:13.212528 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.209779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:43045 (local address 127.5.9.131:46771)
0809 02:10:13.209857 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:13.209869 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:13.209909 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:13.210026 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:13.210027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:13.210028 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:10:13.210079 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:13.210081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.210756 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.210757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:13.211677 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:13.211680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.212061 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.212062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:13.212124 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.212145 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:10:13.212297 (+ 152us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:10:13.212299 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:10:13.212349 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 02:10:13.212394 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 02:10:13.215324 13385 raft_consensus.cc:1261] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Refusing update from remote peer fa59c2c778014739b88a05f5808528fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:13.215350 13234 raft_consensus.cc:1261] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Refusing update from remote peer fa59c2c778014739b88a05f5808528fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:13.215582 13620 consensus_queue.cc:1035] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:13.215713 13630 consensus_queue.cc:1035] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:13.217788 13624 mvcc.cc:205] Tried to move back new op lower bound from 7316444828528390144 to 7316444827928895488. Current Snapshot: MvccSnapshot[applied={T|T < 7316444828528390144}]
I20260809 02:10:13.227460 13060 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:13.227591 13060 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:13.228278 13060 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db: Sending DeleteTablet for 3 replicas of tablet 847ba6c895504761ab34d5ddeaad33bb
I20260809 02:10:13.228605 13363 tablet_service.cc:1514] Processing DeleteTablet for tablet 847ba6c895504761ab34d5ddeaad33bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:13 UTC) from {username='oryx', principal='oryx/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:45372
I20260809 02:10:13.228731 13635 tablet_replica.cc:330] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87: stopping tablet replica
I20260809 02:10:13.228737 13213 tablet_service.cc:1514] Processing DeleteTablet for tablet 847ba6c895504761ab34d5ddeaad33bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:13 UTC) from {username='oryx', principal='oryx/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:59432
I20260809 02:10:13.228801 13635 raft_consensus.cc:2229] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:13.228839 13513 tablet_service.cc:1514] Processing DeleteTablet for tablet 847ba6c895504761ab34d5ddeaad33bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:13 UTC) from {username='oryx', principal='oryx/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:39838
I20260809 02:10:13.228839 13633 tablet_replica.cc:330] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e: stopping tablet replica
I20260809 02:10:13.228886 13635 raft_consensus.cc:2258] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:13.228915 13633 raft_consensus.cc:2229] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:13.228961 13633 raft_consensus.cc:2258] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:13.229043 13634 tablet_replica.cc:330] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb: stopping tablet replica
I20260809 02:10:13.229111 13634 raft_consensus.cc:2229] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:10:13.229178 13634 raft_consensus.cc:2258] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:13.229301 13635 ts_tablet_manager.cc:1905] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:13.229473 13633 ts_tablet_manager.cc:1905] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:13.231256 13634 ts_tablet_manager.cc:1905] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:13.231395 13633 ts_tablet_manager.cc:1918] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:10:13.231457 13633 log.cc:1196] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/wal/wals/847ba6c895504761ab34d5ddeaad33bb
I20260809 02:10:13.231724 13633 ts_tablet_manager.cc:1939] T 847ba6c895504761ab34d5ddeaad33bb P bd321365f16e4e2bb0573bb361226c8e: Deleting consensus metadata
I20260809 02:10:13.232199 13041 catalog_manager.cc:4841] TS bd321365f16e4e2bb0573bb361226c8e (127.5.9.129:43681): tablet 847ba6c895504761ab34d5ddeaad33bb (table test-table [id=1b5262b173e64217857e6020c0e39af5]) successfully deleted
I20260809 02:10:13.232897 13635 ts_tablet_manager.cc:1918] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:10:13.232972 13635 log.cc:1196] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/847ba6c895504761ab34d5ddeaad33bb
I20260809 02:10:13.233158 13634 ts_tablet_manager.cc:1918] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:10:13.233208 13635 ts_tablet_manager.cc:1939] T 847ba6c895504761ab34d5ddeaad33bb P e490ec8957904bd8b266229bee365a87: Deleting consensus metadata
I20260809 02:10:13.233235 13634 log.cc:1196] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/847ba6c895504761ab34d5ddeaad33bb
I20260809 02:10:13.233450 13634 ts_tablet_manager.cc:1939] T 847ba6c895504761ab34d5ddeaad33bb P fa59c2c778014739b88a05f5808528fb: Deleting consensus metadata
I20260809 02:10:13.233564 13042 catalog_manager.cc:4841] TS e490ec8957904bd8b266229bee365a87 (127.5.9.130:40137): tablet 847ba6c895504761ab34d5ddeaad33bb (table test-table [id=1b5262b173e64217857e6020c0e39af5]) successfully deleted
I20260809 02:10:13.233819 13043 catalog_manager.cc:4841] TS fa59c2c778014739b88a05f5808528fb (127.5.9.131:43045): tablet 847ba6c895504761ab34d5ddeaad33bb (table test-table [id=1b5262b173e64217857e6020c0e39af5]) successfully deleted
Aug 09 02:10:13 dist-test-slave-p3t5 krb5kdc[13007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:10:13 dist-test-slave-p3t5 krb5kdc[13007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:13 dist-test-slave-p3t5 krb5kdc[13007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 02:10:13.255275 13113 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:13.248216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38292 (local address 127.5.9.190:43827)
0809 02:10:13.248293 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:13.248295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:13.248431 (+ 136us) server_negotiation.cc:407] Connection header received
0809 02:10:13.248516 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:13.248517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:13.248530 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:13.248590 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:13.249332 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.250216 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.251188 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.251514 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.251595 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.255189 (+ 3594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38292: BlockingRecv error: recv got EOF from 127.0.0.1:38292 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 02:10:13 dist-test-slave-p3t5 krb5kdc[13007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM
I20260809 02:10:13.264680 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.256346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38294 (local address 127.5.9.190:43827)
0809 02:10:13.256401 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:13.256403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:13.256440 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:10:13.256520 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:13.256520 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:13.256529 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:13.256580 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:13.257159 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.257773 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.258757 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.258962 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.259027 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.261818 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:13.261820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:13.261820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:13.261828 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:13.263425 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:13.263796 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:13.263797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:13.263798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:13.263869 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:13.264087 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:13.264088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:13.264088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:13.264216 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:13.264326 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:13.264471 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 02:10:13.264568 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 02:10:13.266446 13060 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:10:13.266647 13060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:10:13.268119 13363 tablet_service.cc:1467] Processing CreateTablet for tablet 019de6ee05e4494e901aa6cb02c18f35 (DEFAULT_TABLE table=test-table [id=b6f1dec92ce34e5a884378d3dd06d3e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:13.268126 13213 tablet_service.cc:1467] Processing CreateTablet for tablet 019de6ee05e4494e901aa6cb02c18f35 (DEFAULT_TABLE table=test-table [id=b6f1dec92ce34e5a884378d3dd06d3e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:13.268187 13513 tablet_service.cc:1467] Processing CreateTablet for tablet 019de6ee05e4494e901aa6cb02c18f35 (DEFAULT_TABLE table=test-table [id=b6f1dec92ce34e5a884378d3dd06d3e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:13.268285 13363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 019de6ee05e4494e901aa6cb02c18f35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:13.268287 13213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 019de6ee05e4494e901aa6cb02c18f35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:13.268298 13513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 019de6ee05e4494e901aa6cb02c18f35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:13.269712 13611 tablet_bootstrap.cc:492] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87: Bootstrap starting.
I20260809 02:10:13.269711 13610 tablet_bootstrap.cc:492] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb: Bootstrap starting.
I20260809 02:10:13.270047 13610 tablet_bootstrap.cc:654] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:13.270165 13611 tablet_bootstrap.cc:654] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:13.270509 13614 tablet_bootstrap.cc:492] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e: Bootstrap starting.
I20260809 02:10:13.270951 13614 tablet_bootstrap.cc:654] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:13.271227 13610 tablet_bootstrap.cc:492] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb: No bootstrap required, opened a new log
I20260809 02:10:13.271287 13610 ts_tablet_manager.cc:1397] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:10:13.271433 13610 raft_consensus.cc:348] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.271546 13610 raft_consensus.cc:374] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:13.271575 13610 raft_consensus.cc:729] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa59c2c778014739b88a05f5808528fb, State: Initialized, Role: FOLLOWER
I20260809 02:10:13.271631 13610 consensus_queue.cc:260] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.271770 13610 ts_tablet_manager.cc:1428] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:13.271989 13611 tablet_bootstrap.cc:492] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87: No bootstrap required, opened a new log
I20260809 02:10:13.272042 13611 ts_tablet_manager.cc:1397] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 02:10:13.272212 13611 raft_consensus.cc:348] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.272316 13611 raft_consensus.cc:374] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:13.272346 13611 raft_consensus.cc:729] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e490ec8957904bd8b266229bee365a87, State: Initialized, Role: FOLLOWER
I20260809 02:10:13.272395 13611 consensus_queue.cc:260] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.272506 13611 ts_tablet_manager.cc:1428] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 02:10:13.272735 13614 tablet_bootstrap.cc:492] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e: No bootstrap required, opened a new log
I20260809 02:10:13.272786 13614 ts_tablet_manager.cc:1397] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 02:10:13.272945 13614 raft_consensus.cc:348] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.273046 13614 raft_consensus.cc:374] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:13.273072 13614 raft_consensus.cc:729] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd321365f16e4e2bb0573bb361226c8e, State: Initialized, Role: FOLLOWER
I20260809 02:10:13.273121 13614 consensus_queue.cc:260] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.273224 13614 ts_tablet_manager.cc:1428] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 02:10:13.628326 13661 raft_consensus.cc:482] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:10:13.628482 13661 raft_consensus.cc:504] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.628782 13661 leader_election.cc:288] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd321365f16e4e2bb0573bb361226c8e (127.5.9.129:43681), e490ec8957904bd8b266229bee365a87 (127.5.9.130:40137)
I20260809 02:10:13.629129 13385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "019de6ee05e4494e901aa6cb02c18f35" candidate_uuid: "fa59c2c778014739b88a05f5808528fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e490ec8957904bd8b266229bee365a87" is_pre_election: true
I20260809 02:10:13.629129 13234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "019de6ee05e4494e901aa6cb02c18f35" candidate_uuid: "fa59c2c778014739b88a05f5808528fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd321365f16e4e2bb0573bb361226c8e" is_pre_election: true
I20260809 02:10:13.629247 13234 raft_consensus.cc:2454] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa59c2c778014739b88a05f5808528fb in term 0.
I20260809 02:10:13.629251 13385 raft_consensus.cc:2454] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa59c2c778014739b88a05f5808528fb in term 0.
I20260809 02:10:13.629452 13462 leader_election.cc:302] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e490ec8957904bd8b266229bee365a87, fa59c2c778014739b88a05f5808528fb; no voters:
I20260809 02:10:13.629580 13661 raft_consensus.cc:2781] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:10:13.629660 13661 raft_consensus.cc:482] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:10:13.629676 13661 raft_consensus.cc:3037] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:13.630501 13661 raft_consensus.cc:504] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.630657 13661 leader_election.cc:288] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [CANDIDATE]: Term 1 election: Requested vote from peers bd321365f16e4e2bb0573bb361226c8e (127.5.9.129:43681), e490ec8957904bd8b266229bee365a87 (127.5.9.130:40137)
I20260809 02:10:13.630925 13385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "019de6ee05e4494e901aa6cb02c18f35" candidate_uuid: "fa59c2c778014739b88a05f5808528fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e490ec8957904bd8b266229bee365a87"
I20260809 02:10:13.631032 13385 raft_consensus.cc:3037] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:13.631001 13234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "019de6ee05e4494e901aa6cb02c18f35" candidate_uuid: "fa59c2c778014739b88a05f5808528fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd321365f16e4e2bb0573bb361226c8e"
I20260809 02:10:13.631151 13234 raft_consensus.cc:3037] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:13.631721 13385 raft_consensus.cc:2454] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa59c2c778014739b88a05f5808528fb in term 1.
I20260809 02:10:13.631855 13234 raft_consensus.cc:2454] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa59c2c778014739b88a05f5808528fb in term 1.
I20260809 02:10:13.631949 13462 leader_election.cc:302] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e490ec8957904bd8b266229bee365a87, fa59c2c778014739b88a05f5808528fb; no voters:
I20260809 02:10:13.632061 13661 raft_consensus.cc:2781] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:13.632146 13661 raft_consensus.cc:686] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [term 1 LEADER]: Becoming Leader. State: Replica: fa59c2c778014739b88a05f5808528fb, State: Running, Role: LEADER
I20260809 02:10:13.632222 13661 consensus_queue.cc:237] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.632874 13060 catalog_manager.cc:5495] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb reported cstate change: term changed from 0 to 1, leader changed from <none> to fa59c2c778014739b88a05f5808528fb (127.5.9.131). New cstate: current_term: 1 leader_uuid: "fa59c2c778014739b88a05f5808528fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } health_report { overall_health: HEALTHY } } }
I20260809 02:10:13.718871 13680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.716463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39874 (local address 127.5.9.131:43045)
0809 02:10:13.716584 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:13.716586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:13.716595 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:10:13.716627 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:13.716627 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:13.716639 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:13.716705 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:10:13.716829 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.717314 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.718193 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.718396 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.718456 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.718634 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:10:13.718671 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:10:13.718729 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:13.718777 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 02:10:13.718801 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":81,"threads_started":1}
I20260809 02:10:13.719869 13234 raft_consensus.cc:1261] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Refusing update from remote peer fa59c2c778014739b88a05f5808528fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:13.719856 13385 raft_consensus.cc:1261] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Refusing update from remote peer fa59c2c778014739b88a05f5808528fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:13.720110 13661 consensus_queue.cc:1035] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:13.720222 13630 consensus_queue.cc:1035] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:13.722949 13060 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:13.723042 13060 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:13.723699 13060 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db: Sending DeleteTablet for 3 replicas of tablet 019de6ee05e4494e901aa6cb02c18f35
I20260809 02:10:13.723943 13213 tablet_service.cc:1514] Processing DeleteTablet for tablet 019de6ee05e4494e901aa6cb02c18f35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:13 UTC) from {username='oryx', principal='oryx/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:59432
I20260809 02:10:13.724027 13513 tablet_service.cc:1514] Processing DeleteTablet for tablet 019de6ee05e4494e901aa6cb02c18f35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:13 UTC) from {username='oryx', principal='oryx/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:39838
I20260809 02:10:13.724018 13363 tablet_service.cc:1514] Processing DeleteTablet for tablet 019de6ee05e4494e901aa6cb02c18f35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:13 UTC) from {username='oryx', principal='oryx/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:45372
I20260809 02:10:13.724049 13633 tablet_replica.cc:330] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e: stopping tablet replica
I20260809 02:10:13.724102 13633 raft_consensus.cc:2229] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:13.724115 13634 tablet_replica.cc:330] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb: stopping tablet replica
I20260809 02:10:13.724141 13633 raft_consensus.cc:2258] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:13.724166 13635 tablet_replica.cc:330] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87: stopping tablet replica
I20260809 02:10:13.724179 13634 raft_consensus.cc:2229] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:10:13.724236 13635 raft_consensus.cc:2229] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:13.724249 13634 raft_consensus.cc:2258] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:13.724288 13635 raft_consensus.cc:2258] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:13.724561 13635 ts_tablet_manager.cc:1905] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:13.724562 13633 ts_tablet_manager.cc:1905] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:13.724598 13634 ts_tablet_manager.cc:1905] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:13.724812 13060 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:10:13.724944 13060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:10:13.726137 13213 tablet_service.cc:1467] Processing CreateTablet for tablet ef4dbcb7518a44b19b2ef679cf7d7535 (DEFAULT_TABLE table=test-table [id=b5f305c618b74648ad93813ed6e2a198]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:13.726372 13633 ts_tablet_manager.cc:1918] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:13.726437 13633 log.cc:1196] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-0/wal/wals/019de6ee05e4494e901aa6cb02c18f35
I20260809 02:10:13.726423 13513 tablet_service.cc:1467] Processing CreateTablet for tablet ef4dbcb7518a44b19b2ef679cf7d7535 (DEFAULT_TABLE table=test-table [id=b5f305c618b74648ad93813ed6e2a198]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:13.726605 13635 ts_tablet_manager.cc:1918] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:13.726675 13633 ts_tablet_manager.cc:1939] T 019de6ee05e4494e901aa6cb02c18f35 P bd321365f16e4e2bb0573bb361226c8e: Deleting consensus metadata
I20260809 02:10:13.726632 13363 tablet_service.cc:1467] Processing CreateTablet for tablet ef4dbcb7518a44b19b2ef679cf7d7535 (DEFAULT_TABLE table=test-table [id=b5f305c618b74648ad93813ed6e2a198]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:13.726714 13635 log.cc:1196] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/019de6ee05e4494e901aa6cb02c18f35
I20260809 02:10:13.726785 13363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef4dbcb7518a44b19b2ef679cf7d7535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:13.726845 13213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef4dbcb7518a44b19b2ef679cf7d7535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:13.726981 13635 ts_tablet_manager.cc:1939] T 019de6ee05e4494e901aa6cb02c18f35 P e490ec8957904bd8b266229bee365a87: Deleting consensus metadata
I20260809 02:10:13.727332 13634 ts_tablet_manager.cc:1918] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:13.727336 13042 catalog_manager.cc:4841] TS e490ec8957904bd8b266229bee365a87 (127.5.9.130:40137): tablet 019de6ee05e4494e901aa6cb02c18f35 (table test-table [id=b6f1dec92ce34e5a884378d3dd06d3e4]) successfully deleted
I20260809 02:10:13.727402 13634 log.cc:1196] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/019de6ee05e4494e901aa6cb02c18f35
I20260809 02:10:13.727618 13634 ts_tablet_manager.cc:1939] T 019de6ee05e4494e901aa6cb02c18f35 P fa59c2c778014739b88a05f5808528fb: Deleting consensus metadata
I20260809 02:10:13.727797 13513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef4dbcb7518a44b19b2ef679cf7d7535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:13.727993 13614 tablet_bootstrap.cc:492] T ef4dbcb7518a44b19b2ef679cf7d7535 P bd321365f16e4e2bb0573bb361226c8e: Bootstrap starting.
I20260809 02:10:13.727986 13611 tablet_bootstrap.cc:492] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87: Bootstrap starting.
I20260809 02:10:13.728650 13614 tablet_bootstrap.cc:654] T ef4dbcb7518a44b19b2ef679cf7d7535 P bd321365f16e4e2bb0573bb361226c8e: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:13.728678 13611 tablet_bootstrap.cc:654] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:13.728755 13610 tablet_bootstrap.cc:492] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb: Bootstrap starting.
I20260809 02:10:13.728860 13043 catalog_manager.cc:4841] TS fa59c2c778014739b88a05f5808528fb (127.5.9.131:43045): tablet 019de6ee05e4494e901aa6cb02c18f35 (table test-table [id=b6f1dec92ce34e5a884378d3dd06d3e4]) successfully deleted
I20260809 02:10:13.729108 13610 tablet_bootstrap.cc:654] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:13.729305 13041 catalog_manager.cc:4841] TS bd321365f16e4e2bb0573bb361226c8e (127.5.9.129:43681): tablet 019de6ee05e4494e901aa6cb02c18f35 (table test-table [id=b6f1dec92ce34e5a884378d3dd06d3e4]) successfully deleted
I20260809 02:10:13.729713 13614 tablet_bootstrap.cc:492] T ef4dbcb7518a44b19b2ef679cf7d7535 P bd321365f16e4e2bb0573bb361226c8e: No bootstrap required, opened a new log
I20260809 02:10:13.729777 13614 ts_tablet_manager.cc:1397] T ef4dbcb7518a44b19b2ef679cf7d7535 P bd321365f16e4e2bb0573bb361226c8e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 02:10:13.729943 13611 tablet_bootstrap.cc:492] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87: No bootstrap required, opened a new log
I20260809 02:10:13.729993 13611 ts_tablet_manager.cc:1397] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 02:10:13.729972 13614 raft_consensus.cc:348] T ef4dbcb7518a44b19b2ef679cf7d7535 P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.730041 13614 raft_consensus.cc:374] T ef4dbcb7518a44b19b2ef679cf7d7535 P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:13.730055 13614 raft_consensus.cc:729] T ef4dbcb7518a44b19b2ef679cf7d7535 P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd321365f16e4e2bb0573bb361226c8e, State: Initialized, Role: FOLLOWER
I20260809 02:10:13.730090 13614 consensus_queue.cc:260] T ef4dbcb7518a44b19b2ef679cf7d7535 P bd321365f16e4e2bb0573bb361226c8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.730185 13614 ts_tablet_manager.cc:1428] T ef4dbcb7518a44b19b2ef679cf7d7535 P bd321365f16e4e2bb0573bb361226c8e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:13.730141 13611 raft_consensus.cc:348] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.730216 13611 raft_consensus.cc:374] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:13.730252 13611 raft_consensus.cc:729] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e490ec8957904bd8b266229bee365a87, State: Initialized, Role: FOLLOWER
I20260809 02:10:13.730306 13611 consensus_queue.cc:260] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.730405 13611 ts_tablet_manager.cc:1428] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 02:10:13.730508 13610 tablet_bootstrap.cc:492] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb: No bootstrap required, opened a new log
I20260809 02:10:13.730548 13610 ts_tablet_manager.cc:1397] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:10:13.730636 13610 raft_consensus.cc:348] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.730703 13610 raft_consensus.cc:374] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:13.730728 13610 raft_consensus.cc:729] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa59c2c778014739b88a05f5808528fb, State: Initialized, Role: FOLLOWER
I20260809 02:10:13.730782 13610 consensus_queue.cc:260] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.730868 13610 ts_tablet_manager.cc:1428] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:13.861990 13616 raft_consensus.cc:482] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:10:13.862113 13616 raft_consensus.cc:504] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.862478 13616 leader_election.cc:288] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd321365f16e4e2bb0573bb361226c8e (127.5.9.129:43681), fa59c2c778014739b88a05f5808528fb (127.5.9.131:43045)
I20260809 02:10:13.865741 13680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.862761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:54867 (local address 127.5.9.131:43045)
0809 02:10:13.862816 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:13.862818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:13.862939 (+ 121us) server_negotiation.cc:407] Connection header received
0809 02:10:13.863022 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:13.863023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:13.863052 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:13.863114 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:13.863334 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.863817 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.864923 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.865310 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.865369 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.865379 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:13.865597 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 02:10:13.865667 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 02:10:13.865770 13689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.862788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:44617 (local address 127.5.9.129:43681)
0809 02:10:13.862932 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:13.862934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:13.862944 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:10:13.862975 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:13.862976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:13.862987 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:13.863054 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:13.863280 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.863828 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.864960 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.865334 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.865430 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.865444 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:13.865648 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 02:10:13.865687 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":53,"threads_started":1}
I20260809 02:10:13.865760 13688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.862672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:43045 (local address 127.5.9.130:54867)
0809 02:10:13.862852 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:13.862864 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:13.862935 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:13.863145 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:13.863146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:13.863148 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:13.863254 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:13.863256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.863936 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.863937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:13.864819 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:13.864821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.865429 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.865430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:13.865516 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.865520 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:10:13.865564 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 02:10:13.865646 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":87,"thread_start_us":75,"threads_started":1}
I20260809 02:10:13.865767 13687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.862672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:43681 (local address 127.5.9.130:44617)
0809 02:10:13.862826 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:13.862838 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:13.862898 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:13.863082 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:13.863084 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:13.863086 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:13.863196 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:13.863199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.863975 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.863976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:13.864826 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:13.864828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.865458 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.865460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:13.865522 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.865525 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:10:13.865597 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:10:13.865663 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":34,"thread_start_us":85,"threads_started":1}
I20260809 02:10:13.866075 13234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef4dbcb7518a44b19b2ef679cf7d7535" candidate_uuid: "e490ec8957904bd8b266229bee365a87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd321365f16e4e2bb0573bb361226c8e" is_pre_election: true
I20260809 02:10:13.866204 13234 raft_consensus.cc:2454] T ef4dbcb7518a44b19b2ef679cf7d7535 P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e490ec8957904bd8b266229bee365a87 in term 0.
I20260809 02:10:13.866276 13533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef4dbcb7518a44b19b2ef679cf7d7535" candidate_uuid: "e490ec8957904bd8b266229bee365a87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa59c2c778014739b88a05f5808528fb" is_pre_election: true
I20260809 02:10:13.866412 13312 leader_election.cc:302] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd321365f16e4e2bb0573bb361226c8e, e490ec8957904bd8b266229bee365a87; no voters:
I20260809 02:10:13.866453 13533 raft_consensus.cc:2454] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e490ec8957904bd8b266229bee365a87 in term 0.
I20260809 02:10:13.866550 13616 raft_consensus.cc:2781] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:10:13.866612 13616 raft_consensus.cc:482] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:10:13.866638 13616 raft_consensus.cc:3037] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:13.867393 13616 raft_consensus.cc:504] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.867568 13616 leader_election.cc:288] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [CANDIDATE]: Term 1 election: Requested vote from peers bd321365f16e4e2bb0573bb361226c8e (127.5.9.129:43681), fa59c2c778014739b88a05f5808528fb (127.5.9.131:43045)
I20260809 02:10:13.867779 13234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef4dbcb7518a44b19b2ef679cf7d7535" candidate_uuid: "e490ec8957904bd8b266229bee365a87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd321365f16e4e2bb0573bb361226c8e"
I20260809 02:10:13.867869 13234 raft_consensus.cc:3037] T ef4dbcb7518a44b19b2ef679cf7d7535 P bd321365f16e4e2bb0573bb361226c8e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:13.867828 13533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef4dbcb7518a44b19b2ef679cf7d7535" candidate_uuid: "e490ec8957904bd8b266229bee365a87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa59c2c778014739b88a05f5808528fb"
I20260809 02:10:13.867920 13533 raft_consensus.cc:3037] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:13.868512 13234 raft_consensus.cc:2454] T ef4dbcb7518a44b19b2ef679cf7d7535 P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e490ec8957904bd8b266229bee365a87 in term 1.
I20260809 02:10:13.868603 13533 raft_consensus.cc:2454] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e490ec8957904bd8b266229bee365a87 in term 1.
I20260809 02:10:13.868752 13312 leader_election.cc:302] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd321365f16e4e2bb0573bb361226c8e, e490ec8957904bd8b266229bee365a87; no voters:
I20260809 02:10:13.868844 13616 raft_consensus.cc:2781] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:13.868990 13616 raft_consensus.cc:686] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [term 1 LEADER]: Becoming Leader. State: Replica: e490ec8957904bd8b266229bee365a87, State: Running, Role: LEADER
I20260809 02:10:13.869107 13616 consensus_queue.cc:237] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } }
I20260809 02:10:13.870036 13058 catalog_manager.cc:5495] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e490ec8957904bd8b266229bee365a87 (127.5.9.130). New cstate: current_term: 1 leader_uuid: "e490ec8957904bd8b266229bee365a87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e490ec8957904bd8b266229bee365a87" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 } health_report { overall_health: UNKNOWN } } }
I20260809 02:10:13.883808 13692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.881160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45384 (local address 127.5.9.130:40137)
0809 02:10:13.881337 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:13.881338 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:13.881344 (+ 6us) server_negotiation.cc:407] Connection header received
0809 02:10:13.881381 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:13.881382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:13.881402 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:13.881479 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:10:13.881605 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.882177 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.882923 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.883140 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.883196 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.883393 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:10:13.883504 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:10:13.883562 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:13.883655 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 02:10:13.883673 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":121,"threads_started":1}
I20260809 02:10:13.889382 13680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.886969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:35665 (local address 127.5.9.131:43045)
0809 02:10:13.887036 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:13.887040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:13.887079 (+ 39us) server_negotiation.cc:407] Connection header received
0809 02:10:13.887137 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:13.887137 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:13.887148 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:13.887202 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:10:13.887388 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.887875 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.888663 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.888871 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.888931 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.889150 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:10:13.889183 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:10:13.889234 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:13.889289 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 02:10:13.889315 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:10:13.889395 13695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.886844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:43045 (local address 127.5.9.130:35665)
0809 02:10:13.887013 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:13.887025 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:13.887092 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:13.887238 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:13.887239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:13.887241 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:10:13.887308 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:13.887310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.887956 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.887956 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:13.888575 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:13.888576 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.888973 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.888973 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:13.889031 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.889047 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:10:13.889248 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:10:13.889249 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 02:10:13.889285 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 02:10:13.889323 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260809 02:10:13.892071 13533 raft_consensus.cc:1261] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb [term 1 FOLLOWER]: Refusing update from remote peer e490ec8957904bd8b266229bee365a87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:13.892253 13234 raft_consensus.cc:1261] T ef4dbcb7518a44b19b2ef679cf7d7535 P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Refusing update from remote peer e490ec8957904bd8b266229bee365a87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:13.892423 13616 consensus_queue.cc:1035] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa59c2c778014739b88a05f5808528fb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 43045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:13.892575 13690 consensus_queue.cc:1035] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd321365f16e4e2bb0573bb361226c8e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 43681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:13.900321 13692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.897862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:45473 (local address 127.5.9.130:40137)
0809 02:10:13.897912 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:13.897914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:13.898041 (+ 127us) server_negotiation.cc:407] Connection header received
0809 02:10:13.898111 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:13.898112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:13.898124 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:13.898174 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:10:13.898303 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.898797 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.899549 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.899830 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.899904 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.900063 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:10:13.900098 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:10:13.900154 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:13.900218 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 02:10:13.900239 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:10:13.900345 13700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.897809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:40137 (local address 127.5.9.131:45473)
0809 02:10:13.897958 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:13.897977 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:13.898058 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:13.898175 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:13.898176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:13.898178 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:10:13.898234 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:13.898235 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.898882 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.898883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:13.899459 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:13.899460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:13.899906 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:13.899907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:13.899967 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:13.899973 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:10:13.900165 (+ 192us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:10:13.900168 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:10:13.900206 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 02:10:13.900265 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":68,"threads_started":1}
I20260809 02:10:13.910491 13058 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:13.910637 13058 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:13.911435 13058 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63b785ddfae2467ebf214a4ab87139db: Sending DeleteTablet for 3 replicas of tablet ef4dbcb7518a44b19b2ef679cf7d7535
I20260809 02:10:13.911739 13213 tablet_service.cc:1514] Processing DeleteTablet for tablet ef4dbcb7518a44b19b2ef679cf7d7535 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:13 UTC) from {username='oryx', principal='oryx/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:59432
I20260809 02:10:13.911764 13363 tablet_service.cc:1514] Processing DeleteTablet for tablet ef4dbcb7518a44b19b2ef679cf7d7535 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:13 UTC) from {username='oryx', principal='oryx/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:45372
I20260809 02:10:13.911890 13635 tablet_replica.cc:330] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87: stopping tablet replica
I20260809 02:10:13.911959 13635 raft_consensus.cc:2229] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:10:13.912052 13635 raft_consensus.cc:2258] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:13.912149 13633 tablet_replica.cc:330] T ef4dbcb7518a44b19b2ef679cf7d7535 P bd321365f16e4e2bb0573bb361226c8e: stopping tablet replica
I20260809 02:10:13.912194 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 13140
I20260809 02:10:13.912220 13633 raft_consensus.cc:2229] T ef4dbcb7518a44b19b2ef679cf7d7535 P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:13.912266 13633 raft_consensus.cc:2258] T ef4dbcb7518a44b19b2ef679cf7d7535 P bd321365f16e4e2bb0573bb361226c8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:13.912684 13635 ts_tablet_manager.cc:1905] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:13.914469 13635 ts_tablet_manager.cc:1918] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:10:13.914527 13635 log.cc:1196] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/ef4dbcb7518a44b19b2ef679cf7d7535
I20260809 02:10:13.914753 13635 ts_tablet_manager.cc:1939] T ef4dbcb7518a44b19b2ef679cf7d7535 P e490ec8957904bd8b266229bee365a87: Deleting consensus metadata
I20260809 02:10:13.914943 13513 tablet_service.cc:1514] Processing DeleteTablet for tablet ef4dbcb7518a44b19b2ef679cf7d7535 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:13 UTC) from {username='oryx', principal='oryx/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:39838
I20260809 02:10:13.915069 13634 tablet_replica.cc:330] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb: stopping tablet replica
I20260809 02:10:13.915139 13634 raft_consensus.cc:2229] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:13.915192 13634 raft_consensus.cc:2258] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:13.915547 13042 catalog_manager.cc:4841] TS e490ec8957904bd8b266229bee365a87 (127.5.9.130:40137): tablet ef4dbcb7518a44b19b2ef679cf7d7535 (table test-table [id=b5f305c618b74648ad93813ed6e2a198]) successfully deleted
I20260809 02:10:13.918488 13634 ts_tablet_manager.cc:1905] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:13.920262 13634 ts_tablet_manager.cc:1918] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:10:13.920342 13634 log.cc:1196] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/ef4dbcb7518a44b19b2ef679cf7d7535
I20260809 02:10:13.920573 13634 ts_tablet_manager.cc:1939] T ef4dbcb7518a44b19b2ef679cf7d7535 P fa59c2c778014739b88a05f5808528fb: Deleting consensus metadata
I20260809 02:10:13.921114 13043 catalog_manager.cc:4841] TS fa59c2c778014739b88a05f5808528fb (127.5.9.131:43045): tablet ef4dbcb7518a44b19b2ef679cf7d7535 (table test-table [id=b5f305c618b74648ad93813ed6e2a198]) successfully deleted
W20260809 02:10:13.923689 13041 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:43681 (error 108)
I20260809 02:10:13.924398 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:13.924165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:43681 (local address 127.0.0.1:59448)
0809 02:10:13.924283 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:13.924333 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260809 02:10:13.924445 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 13294
W20260809 02:10:13.927119 13041 catalog_manager.cc:4584] TS bd321365f16e4e2bb0573bb361226c8e (127.5.9.129:43681): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ef4dbcb7518a44b19b2ef679cf7d7535: Network error: Client connection negotiation failed: client connection to 127.5.9.129:43681: connect: Connection refused (error 111)
I20260809 02:10:13.932862 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 13443
I20260809 02:10:13.942194 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 13021
2026-08-09T02:10:13Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3152 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:10:14 dist-test-slave-p3t5 krb5kdc[13708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:10:14 dist-test-slave-p3t5 krb5kdc[13708](info): set up 2 sockets
Aug 09 02:10:14 dist-test-slave-p3t5 krb5kdc[13708](info): commencing operation
krb5kdc: starting...
W20260809 02:10:15.022907 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:10:15 dist-test-slave-p3t5 krb5kdc[13708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:10:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:10:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:10:15.157531 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:45801
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:45801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:15.242647 13725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:15.242954 13725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:15.242997 13725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:15.244222 13725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:15.244308 13725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:15.244334 13725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:15.244356 13725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:15.244386 13725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:10:15.246034 13725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:45801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:45801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:15.246337 13725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:15.246587 13725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:15.251658 13732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:15.251704 13730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:15.251706 13734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:15.252045 13725 server_base.cc:1034] running on GCE node
I20260809 02:10:15.252391 13725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:15.252699 13725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:15.253897 13725 hybrid_clock.cc:648] HybridClock initialized: now 1786241415253879 us; error 55 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T02:10:15Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1310 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:10:15 dist-test-slave-p3t5 krb5kdc[13740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:10:15 dist-test-slave-p3t5 krb5kdc[13740](info): set up 2 sockets
Aug 09 02:10:15 dist-test-slave-p3t5 krb5kdc[13740](info): commencing operation
krb5kdc: starting...
W20260809 02:10:16.346980 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:10:16 dist-test-slave-p3t5 krb5kdc[13740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:10:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:10:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:10:16.484807 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:36009
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:36009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:16.575644 13755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:16.575898 13755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:16.575924 13755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:16.577096 13755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:16.577135 13755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:16.577150 13755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:16.577163 13755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:16.577175 13755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:10:16.578768 13755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:36009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:36009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:16.579006 13755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:16.579211 13755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:16.584415 13763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:16.584388 13761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:16.584482 13755 server_base.cc:1034] running on GCE node
W20260809 02:10:16.584411 13760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:16.584933 13755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:16.585244 13755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:16.586403 13755 hybrid_clock.cc:648] HybridClock initialized: now 1786241416586386 us; error 51 us; skew 500 ppm
Aug 09 02:10:16 dist-test-slave-p3t5 krb5kdc[13740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241416, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:16.594271 13755 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:10:16.594851 13755 webserver.cc:460] Webserver started at http://127.5.9.190:35533/ using document root <none> and password file <none>
I20260809 02:10:16.595135 13755 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:16.595211 13755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:16.595388 13755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:16.596616 13755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "1c98f76a9b304f6dbdf9dea5323364ea"
format_stamp: "Formatted at 2026-08-09 02:10:16 on dist-test-slave-p3t5"
server_key: "c28137a9e4249ba44ace8a890d665af8"
server_key_iv: "568afad97b40ca7bdefab4646232188d"
server_key_version: "encryptionkey@0"
I20260809 02:10:16.597018 13755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "1c98f76a9b304f6dbdf9dea5323364ea"
format_stamp: "Formatted at 2026-08-09 02:10:16 on dist-test-slave-p3t5"
server_key: "c28137a9e4249ba44ace8a890d665af8"
server_key_iv: "568afad97b40ca7bdefab4646232188d"
server_key_version: "encryptionkey@0"
I20260809 02:10:16.598701 13755 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260809 02:10:16.599596 13770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:16.599812 13755 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:10:16.599933 13755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "1c98f76a9b304f6dbdf9dea5323364ea"
format_stamp: "Formatted at 2026-08-09 02:10:16 on dist-test-slave-p3t5"
server_key: "c28137a9e4249ba44ace8a890d665af8"
server_key_iv: "568afad97b40ca7bdefab4646232188d"
server_key_version: "encryptionkey@0"
I20260809 02:10:16.600018 13755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:16.620720 13755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:16.623497 13755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:16.623685 13755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:16.628425 13755 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:36009
I20260809 02:10:16.628489 13827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:36009 every 8 connection(s)
I20260809 02:10:16.628911 13755 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:10:16.629858 13828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:16.631369 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 13755
I20260809 02:10:16.631472 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:10:16.631687 5158 external_mini_cluster.cc:1428] Setting key e8ab1d83ce0eb18e60e4a0a3274c70d2
I20260809 02:10:16.632905 13828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea: Bootstrap starting.
I20260809 02:10:16.633749 13828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:16.634079 13828 log.cc:824] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:16.634969 13828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea: No bootstrap required, opened a new log
I20260809 02:10:16.637346 13828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c98f76a9b304f6dbdf9dea5323364ea" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36009 } }
I20260809 02:10:16.637575 13828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:16.637622 13828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c98f76a9b304f6dbdf9dea5323364ea, State: Initialized, Role: FOLLOWER
I20260809 02:10:16.637795 13828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c98f76a9b304f6dbdf9dea5323364ea" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36009 } }
I20260809 02:10:16.637907 13828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:10:16.637946 13828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:10:16.638001 13828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:16.638636 13828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c98f76a9b304f6dbdf9dea5323364ea" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36009 } }
I20260809 02:10:16.638767 13828 leader_election.cc:302] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c98f76a9b304f6dbdf9dea5323364ea; no voters:
I20260809 02:10:16.638969 13828 leader_election.cc:288] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:10:16.639022 13834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:16.639242 13834 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 1 LEADER]: Becoming Leader. State: Replica: 1c98f76a9b304f6dbdf9dea5323364ea, State: Running, Role: LEADER
I20260809 02:10:16.639292 13828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:16.639451 13834 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c98f76a9b304f6dbdf9dea5323364ea" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36009 } }
Aug 09 02:10:16 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:10:16.639955 13836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c98f76a9b304f6dbdf9dea5323364ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c98f76a9b304f6dbdf9dea5323364ea" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36009 } } }
I20260809 02:10:16.640064 13836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:16.640419 13843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:10:16.640897 13835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c98f76a9b304f6dbdf9dea5323364ea. Latest consensus state: current_term: 1 leader_uuid: "1c98f76a9b304f6dbdf9dea5323364ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c98f76a9b304f6dbdf9dea5323364ea" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36009 } } }
I20260809 02:10:16.641026 13843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:10:16.641043 13835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:16.642927 13843 catalog_manager.cc:1347] Generated new cluster ID: db63c5d6caf943f6ba2ed067814a7624
I20260809 02:10:16.642989 13843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:10:16.644610 13832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:16.632357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37174 (local address 127.5.9.190:36009)
0809 02:10:16.632568 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:16.632579 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:16.632614 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:10:16.633194 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:16.633203 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:16.633279 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:16.633453 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:16.634234 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:16.635651 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:16.636461 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:16.637198 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:16.637319 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:16.640378 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:16.640382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:16.640382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:16.640396 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:16.642790 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:16.643261 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:16.643264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:16.643265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:16.643380 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:16.643603 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:16.643605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:16.643605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:16.643935 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:16.644087 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:16.644206 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 02:10:16.644375 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":87,"threads_started":1}
I20260809 02:10:16.647023 13843 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:10:16.647538 13843 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:10:16.651132 13843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea: Generated new TSK 0
I20260809 02:10:16.651345 13843 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:10:16.689715 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:36009
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:16.816682 13856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:16.817009 13856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:16.817049 13856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:16.818990 13856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:16.819072 13856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:16.819115 13856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:10:16.821496 13856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:36009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:16.821851 13856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:16.822141 13856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:16.828011 13864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:16.828054 13861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:16.828116 13862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:16.828441 13856 server_base.cc:1034] running on GCE node
I20260809 02:10:16.828711 13856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:16.828989 13856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:16.830158 13856 hybrid_clock.cc:648] HybridClock initialized: now 1786241416830131 us; error 46 us; skew 500 ppm
Aug 09 02:10:16 dist-test-slave-p3t5 krb5kdc[13740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241416, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:16.833040 13856 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:10:16.833539 13856 webserver.cc:460] Webserver started at http://127.5.9.129:42423/ using document root <none> and password file <none>
I20260809 02:10:16.833806 13856 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:16.833895 13856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:16.834051 13856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:16.835182 13856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "32dc0e91678546ecaded3bc5d421e4b3"
format_stamp: "Formatted at 2026-08-09 02:10:16 on dist-test-slave-p3t5"
server_key: "e1cf761c6b609951f646c611c633910d"
server_key_iv: "fd2db8d8feef4a17f1d69293bd2f4b2f"
server_key_version: "encryptionkey@0"
I20260809 02:10:16.835578 13856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "32dc0e91678546ecaded3bc5d421e4b3"
format_stamp: "Formatted at 2026-08-09 02:10:16 on dist-test-slave-p3t5"
server_key: "e1cf761c6b609951f646c611c633910d"
server_key_iv: "fd2db8d8feef4a17f1d69293bd2f4b2f"
server_key_version: "encryptionkey@0"
I20260809 02:10:16.837065 13856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:10:16.838056 13872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:16.838256 13856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:16.838367 13856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "32dc0e91678546ecaded3bc5d421e4b3"
format_stamp: "Formatted at 2026-08-09 02:10:16 on dist-test-slave-p3t5"
server_key: "e1cf761c6b609951f646c611c633910d"
server_key_iv: "fd2db8d8feef4a17f1d69293bd2f4b2f"
server_key_version: "encryptionkey@0"
I20260809 02:10:16.838454 13856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:16.849526 13856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:16.852094 13856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:16.852280 13856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:16.852548 13856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:16.852902 13856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:16.852962 13856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:16.853009 13856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:16.853040 13856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:16.859403 13856 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:33221
I20260809 02:10:16.859453 13996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:33221 every 8 connection(s)
I20260809 02:10:16.859793 13856 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:10:16 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241416, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:10:16.867854 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 13856
I20260809 02:10:16.868005 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:10:16.868257 5158 external_mini_cluster.cc:1428] Setting key cbe55c36414ab37bdc6cec3bec19bb27
I20260809 02:10:16.871079 13832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:16.860592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:53417 (local address 127.5.9.190:36009)
0809 02:10:16.860699 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:16.860702 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:16.860914 (+ 212us) server_negotiation.cc:407] Connection header received
0809 02:10:16.861534 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:16.861535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:16.861550 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:16.861604 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:16.862793 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:16.863368 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:16.864531 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:16.864786 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:16.864890 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:16.867455 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:16.867456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:16.867456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:16.867464 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:16.869381 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:16.869933 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:16.869935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:16.869935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:16.870010 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:16.870254 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:16.870255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:16.870255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:16.870421 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:16.870491 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:16.870825 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 02:10:16.870937 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 02:10:16.871254 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:16.860504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:53417)
0809 02:10:16.860750 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:16.860807 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:16.861403 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:16.861712 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:16.861716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:16.862425 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:16.862645 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:16.862650 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:16.863483 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:16.863485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:16.864419 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:16.864422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:16.864905 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:16.864906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:16.865050 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:16.865764 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:16.865773 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:16.867348 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:16.869466 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:16.869474 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:16.869475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:16.869849 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:16.870077 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:16.870078 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:16.870078 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:16.870184 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:16.870509 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:16.870517 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:16.870668 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 02:10:16.870762 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 02:10:16.870917 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":112,"threads_started":1}
I20260809 02:10:16.871882 13997 heartbeater.cc:347] Connected to a master server at 127.5.9.190:36009
I20260809 02:10:16.872010 13997 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:16.872231 13997 heartbeater.cc:511] Master 127.5.9.190:36009 requested a full tablet report, sending...
I20260809 02:10:16.872788 13788 ts_manager.cc:194] Registered new tserver with Master: 32dc0e91678546ecaded3bc5d421e4b3 (127.5.9.129:33221)
I20260809 02:10:16.873457 13788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:53417
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:10:16.901338 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:36009
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:16.988399 14002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:16.988690 14002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:16.988731 14002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:16.990010 14002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:16.990089 14002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:16.990135 14002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:10:16.991719 14002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:36009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:16.992031 14002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:16.992306 14002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:10:16.997879 14002 server_base.cc:1034] running on GCE node
W20260809 02:10:16.997885 14010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:16.997885 14008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:16.997895 14007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:16.998451 14002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:16.998741 14002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:16.999930 14002 hybrid_clock.cc:648] HybridClock initialized: now 1786241416999919 us; error 34 us; skew 500 ppm
Aug 09 02:10:16 dist-test-slave-p3t5 krb5kdc[13740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241416, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:17.003141 14002 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:10:17.003759 14002 webserver.cc:460] Webserver started at http://127.5.9.130:41413/ using document root <none> and password file <none>
I20260809 02:10:17.004046 14002 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:17.004124 14002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:17.004300 14002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:17.005468 14002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "6fdf16f227f642318528ad8b83c3c476"
format_stamp: "Formatted at 2026-08-09 02:10:16 on dist-test-slave-p3t5"
server_key: "2a8fe90067f3e63c5ded00e79e33af1e"
server_key_iv: "e0fb7339630a989d94e787bd22c71726"
server_key_version: "encryptionkey@0"
I20260809 02:10:17.006142 14002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "6fdf16f227f642318528ad8b83c3c476"
format_stamp: "Formatted at 2026-08-09 02:10:16 on dist-test-slave-p3t5"
server_key: "2a8fe90067f3e63c5ded00e79e33af1e"
server_key_iv: "e0fb7339630a989d94e787bd22c71726"
server_key_version: "encryptionkey@0"
I20260809 02:10:17.007845 14002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:10:17.008824 14017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:17.009038 14002 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:17.009161 14002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "6fdf16f227f642318528ad8b83c3c476"
format_stamp: "Formatted at 2026-08-09 02:10:16 on dist-test-slave-p3t5"
server_key: "2a8fe90067f3e63c5ded00e79e33af1e"
server_key_iv: "e0fb7339630a989d94e787bd22c71726"
server_key_version: "encryptionkey@0"
I20260809 02:10:17.009249 14002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:17.030781 14002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:17.033330 14002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:17.033543 14002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:17.033874 14002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:17.034243 14002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:17.034312 14002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:17.034363 14002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:17.034389 14002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:17.040904 14002 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:40755
I20260809 02:10:17.040959 14135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:40755 every 8 connection(s)
I20260809 02:10:17.041338 14002 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:10:17.048228 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 14002
I20260809 02:10:17.048341 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241416, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:10:17.048625 5158 external_mini_cluster.cc:1428] Setting key 00a5c32a4dd9cc1677c72acdb4198534
I20260809 02:10:17.052537 13832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:17.042403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:38259 (local address 127.5.9.190:36009)
0809 02:10:17.042484 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:17.042486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:17.042627 (+ 141us) server_negotiation.cc:407] Connection header received
0809 02:10:17.043284 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:17.043285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:17.043297 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:17.043361 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:17.044443 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.044957 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.046035 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.046280 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.046353 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.049042 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:17.049044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:17.049045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:17.049055 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:17.050889 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:17.051355 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:17.051356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:17.051357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:17.051432 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:17.051807 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:17.051808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:17.051808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:17.051942 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:17.052001 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:17.052380 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 02:10:17.052466 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:10:17.052842 14138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:17.042285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:38259)
0809 02:10:17.042476 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:17.042528 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:17.043151 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:17.043448 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:17.043451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:17.044105 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:17.044316 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.044322 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.045047 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.045048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.045913 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.045915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.046401 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.046402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.046530 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.047196 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:17.047206 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:17.048946 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:17.050992 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:17.051001 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:17.051003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:17.051265 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:17.051518 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:17.051519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:17.051519 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:17.051727 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:17.052015 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:17.052023 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:17.052219 (+ 196us) client_negotiation.cc:769] Sending connection context
0809 02:10:17.052350 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 02:10:17.052567 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":71,"threads_started":1}
I20260809 02:10:17.053431 14136 heartbeater.cc:347] Connected to a master server at 127.5.9.190:36009
I20260809 02:10:17.053555 14136 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:17.053791 14136 heartbeater.cc:511] Master 127.5.9.190:36009 requested a full tablet report, sending...
I20260809 02:10:17.054302 13788 ts_manager.cc:194] Registered new tserver with Master: 6fdf16f227f642318528ad8b83c3c476 (127.5.9.130:40755)
I20260809 02:10:17.054716 13788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:38259
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:10:17.081453 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:36009
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:17.169713 14141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:17.169962 14141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:17.169987 14141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:17.171168 14141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:17.171207 14141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:17.171234 14141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:10:17.172645 14141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:36009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:17.172857 14141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:17.173058 14141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:17.178439 14148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:17.178408 14147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:17.178418 14150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:17.178730 14141 server_base.cc:1034] running on GCE node
I20260809 02:10:17.178997 14141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:17.179271 14141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:17.180460 14141 hybrid_clock.cc:648] HybridClock initialized: now 1786241417180400 us; error 71 us; skew 500 ppm
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241417, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:17.183416 14141 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:10:17.183995 14141 webserver.cc:460] Webserver started at http://127.5.9.131:41417/ using document root <none> and password file <none>
I20260809 02:10:17.184248 14141 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:17.184337 14141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:17.184525 14141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:17.185755 14141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "d39c9d5bc01f4923953b5c7de5c99c66"
format_stamp: "Formatted at 2026-08-09 02:10:17 on dist-test-slave-p3t5"
server_key: "049ef09f76ed9771775bc33eb1d37307"
server_key_iv: "889da130022437fd07e838f212409880"
server_key_version: "encryptionkey@0"
I20260809 02:10:17.186174 14141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "d39c9d5bc01f4923953b5c7de5c99c66"
format_stamp: "Formatted at 2026-08-09 02:10:17 on dist-test-slave-p3t5"
server_key: "049ef09f76ed9771775bc33eb1d37307"
server_key_iv: "889da130022437fd07e838f212409880"
server_key_version: "encryptionkey@0"
I20260809 02:10:17.187623 14141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:10:17.188432 14157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:17.188640 14141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:17.188752 14141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "d39c9d5bc01f4923953b5c7de5c99c66"
format_stamp: "Formatted at 2026-08-09 02:10:17 on dist-test-slave-p3t5"
server_key: "049ef09f76ed9771775bc33eb1d37307"
server_key_iv: "889da130022437fd07e838f212409880"
server_key_version: "encryptionkey@0"
I20260809 02:10:17.188833 14141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:17.202327 14141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:17.204965 14141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:17.205124 14141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:17.205395 14141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:17.205801 14141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:17.205842 14141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:17.205868 14141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:17.205881 14141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:17.212361 14141 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:45943
I20260809 02:10:17.212402 14277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:45943 every 8 connection(s)
I20260809 02:10:17.212792 14141 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:10:17.218355 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 14141
I20260809 02:10:17.218509 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:10:17.218705 5158 external_mini_cluster.cc:1428] Setting key 2eb4dab55cc7bd5b5d71e9149bf9592d
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241417, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:10:17.223846 13832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:17.213700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:59591 (local address 127.5.9.190:36009)
0809 02:10:17.213766 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:17.213768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:17.214010 (+ 242us) server_negotiation.cc:407] Connection header received
0809 02:10:17.214575 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:17.214577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:17.214590 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:17.214647 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:17.215678 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.216198 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.217241 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.217443 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.217526 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.220620 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:17.220622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:17.220623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:17.220631 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:17.222282 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:17.222763 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:17.222764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:17.222765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:17.222837 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:17.223116 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:17.223117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:17.223117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:17.223261 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:17.223339 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:17.223645 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 02:10:17.223746 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:10:17.224093 14280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:17.213658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:59591)
0809 02:10:17.213867 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:17.213911 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:17.214458 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:17.214730 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:17.214733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:17.215357 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:17.215562 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.215567 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.216307 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.216308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.217139 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.217141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.217536 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.217537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.217725 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.218335 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:17.218348 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:17.220513 (+ 2165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:17.222382 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:17.222391 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:17.222394 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:17.222663 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:17.222913 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:17.222914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:17.222914 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:17.223037 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:17.223329 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:17.223337 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:17.223498 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 02:10:17.223623 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 02:10:17.223809 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
I20260809 02:10:17.224674 14278 heartbeater.cc:347] Connected to a master server at 127.5.9.190:36009
I20260809 02:10:17.224841 14278 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:17.225062 14278 heartbeater.cc:511] Master 127.5.9.190:36009 requested a full tablet report, sending...
I20260809 02:10:17.225525 13788 ts_manager.cc:194] Registered new tserver with Master: d39c9d5bc01f4923953b5c7de5c99c66 (127.5.9.131:45943)
I20260809 02:10:17.225982 13788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:59591
I20260809 02:10:17.231066 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.129.keytab.
Entry for principal oryx/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.129.keytab.
WARNING: no policy specified for oryx/127.5.9.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.130.keytab.
Entry for principal oryx/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.130.keytab.
WARNING: no policy specified for oryx/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.131.keytab.
Entry for principal oryx/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.131.keytab.
I20260809 02:10:17.322413 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 13856
I20260809 02:10:17.330017 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 14002
I20260809 02:10:17.335918 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 14141
I20260809 02:10:17.343807 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 13755
I20260809 02:10:17.353446 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:36009
--webserver_interface=127.5.9.190
--webserver_port=35533
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:36009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:17.441008 14287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:17.441241 14287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:17.441263 14287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:17.442569 14287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:17.442619 14287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:17.442634 14287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:17.442647 14287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:17.442659 14287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:10:17.444236 14287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:36009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:36009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=35533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:17.444527 14287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:17.444788 14287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:17.449941 14293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:17.449988 14292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:17.450217 14287 server_base.cc:1034] running on GCE node
W20260809 02:10:17.449954 14295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:17.450560 14287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:17.450917 14287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:17.452070 14287 hybrid_clock.cc:648] HybridClock initialized: now 1786241417452052 us; error 46 us; skew 500 ppm
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241417, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:17.455034 14287 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:10:17.455595 14287 webserver.cc:460] Webserver started at http://127.5.9.190:35533/ using document root <none> and password file <none>
I20260809 02:10:17.455900 14287 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:17.456001 14287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:17.457523 14287 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:10:17.458325 14302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:17.458536 14287 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:10:17.458653 14287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "1c98f76a9b304f6dbdf9dea5323364ea"
format_stamp: "Formatted at 2026-08-09 02:10:16 on dist-test-slave-p3t5"
server_key: "c28137a9e4249ba44ace8a890d665af8"
server_key_iv: "568afad97b40ca7bdefab4646232188d"
server_key_version: "encryptionkey@0"
I20260809 02:10:17.459095 14287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:17.478615 14287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:17.481105 14287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:17.481297 14287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:17.486006 14287 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:36009
I20260809 02:10:17.486100 14358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:36009 every 8 connection(s)
I20260809 02:10:17.486511 14287 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:10:17.488128 14359 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:10:17.488845 14359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea: Bootstrap starting.
I20260809 02:10:17.490545 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 14287
I20260809 02:10:17.490922 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:33221
--local_ip_for_outbound_sockets=127.5.9.129
--tserver_master_addrs=127.5.9.190:36009
--webserver_port=42423
--webserver_interface=127.5.9.129
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.9.129
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.129.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:10:17.498719 14359 log.cc:824] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:17.500584 14359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:10:17.500923 14359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea: Bootstrap complete.
I20260809 02:10:17.503398 14359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c98f76a9b304f6dbdf9dea5323364ea" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36009 } }
I20260809 02:10:17.503643 14359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c98f76a9b304f6dbdf9dea5323364ea, State: Initialized, Role: FOLLOWER
I20260809 02:10:17.503824 14359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c98f76a9b304f6dbdf9dea5323364ea" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36009 } }
I20260809 02:10:17.503911 14359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:10:17.503983 14359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:10:17.504031 14359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:10:17.505120 14359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c98f76a9b304f6dbdf9dea5323364ea" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36009 } }
I20260809 02:10:17.505215 14359 leader_election.cc:302] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c98f76a9b304f6dbdf9dea5323364ea; no voters:
I20260809 02:10:17.505465 14359 leader_election.cc:288] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 02:10:17.505534 14363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:10:17.505728 14363 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [term 2 LEADER]: Becoming Leader. State: Replica: 1c98f76a9b304f6dbdf9dea5323364ea, State: Running, Role: LEADER
I20260809 02:10:17.505810 14359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:17.505836 14363 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c98f76a9b304f6dbdf9dea5323364ea" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36009 } }
I20260809 02:10:17.506161 14363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c98f76a9b304f6dbdf9dea5323364ea. Latest consensus state: current_term: 2 leader_uuid: "1c98f76a9b304f6dbdf9dea5323364ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c98f76a9b304f6dbdf9dea5323364ea" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36009 } } }
I20260809 02:10:17.506268 14363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:17.506165 14364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1c98f76a9b304f6dbdf9dea5323364ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c98f76a9b304f6dbdf9dea5323364ea" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 36009 } } }
I20260809 02:10:17.506758 14364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:17.507853 14379 catalog_manager.cc:1259] Loaded cluster ID: db63c5d6caf943f6ba2ed067814a7624
I20260809 02:10:17.507918 14379 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea: loading cluster ID for follower catalog manager: success
I20260809 02:10:17.509104 14379 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea: acquiring CA information for follower catalog manager: success
I20260809 02:10:17.509470 14379 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:10:17.509549 14368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:10:17.509765 14368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:10:17.509850 14368 catalog_manager.cc:1259] Loaded cluster ID: db63c5d6caf943f6ba2ed067814a7624
I20260809 02:10:17.509884 14368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:10:17.510067 14368 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:10:17.510155 14368 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1c98f76a9b304f6dbdf9dea5323364ea: Loaded TSK: 0
I20260809 02:10:17.510285 14368 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:10:17.579443 14361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:17.579721 14361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:17.579767 14361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:17.581014 14361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:17.581089 14361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:17.581136 14361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:10:17.582749 14361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.129.keytab
--principal=oryx/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:33221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=42423
--tserver_master_addrs=127.5.9.190:36009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:17.583079 14361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:17.583348 14361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:17.589673 14386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:17.589704 14385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:17.589735 14388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:17.590013 14361 server_base.cc:1034] running on GCE node
I20260809 02:10:17.590232 14361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:17.590493 14361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:17.591686 14361 hybrid_clock.cc:648] HybridClock initialized: now 1786241417591670 us; error 32 us; skew 500 ppm
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241417, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:17.594725 14361 init.cc:377] Logged in from keytab as oryx/127.5.9.129@KRBTEST.COM (short username oryx)
I20260809 02:10:17.595232 14361 webserver.cc:460] Webserver started at http://127.5.9.129:42423/ using document root <none> and password file <none>
I20260809 02:10:17.595491 14361 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:17.595587 14361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:17.597169 14361 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:10:17.597918 14395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:17.598099 14361 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:10:17.598212 14361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "32dc0e91678546ecaded3bc5d421e4b3"
format_stamp: "Formatted at 2026-08-09 02:10:16 on dist-test-slave-p3t5"
server_key: "e1cf761c6b609951f646c611c633910d"
server_key_iv: "fd2db8d8feef4a17f1d69293bd2f4b2f"
server_key_version: "encryptionkey@0"
I20260809 02:10:17.598605 14361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:17.618780 14361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:17.621403 14361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:17.621593 14361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:17.621893 14361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:17.622238 14361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:17.622303 14361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:17.622351 14361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:17.622376 14361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:17.629290 14361 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:33221
I20260809 02:10:17.629364 14513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:33221 every 8 connection(s)
I20260809 02:10:17.629832 14361 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:10:17.637917 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 14361
I20260809 02:10:17.638345 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:40755
--local_ip_for_outbound_sockets=127.5.9.130
--tserver_master_addrs=127.5.9.190:36009
--webserver_port=41413
--webserver_interface=127.5.9.130
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.9.130
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.130.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:17.640761 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:17.630608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:51771)
0809 02:10:17.630851 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:17.630904 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:17.631546 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:17.632041 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:17.632045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:17.632907 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:17.633197 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.633206 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.634448 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.634450 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.635824 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.635828 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.636493 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.636494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.636639 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.637308 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:17.637318 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:17.640509 (+ 3191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":129,"threads_started":1}
W20260809 02:10:17.640951 14517 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:17.630770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:51771 (local address 127.5.9.190:36009)
0809 02:10:17.630953 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:17.630962 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:17.631005 (+ 43us) server_negotiation.cc:407] Connection header received
0809 02:10:17.631760 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:17.631766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:17.631845 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:17.631958 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:17.633338 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.634327 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.635950 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.636383 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.636482 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.640666 (+ 4184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:51771: BlockingRecv error: recv got EOF from 127.5.9.129:51771 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
W20260809 02:10:17.640997 14514 heartbeater.cc:644] Failed to heartbeat to 127.5.9.190:36009 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.9.190:36009: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:17.646104 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:17.641251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:54659)
0809 02:10:17.641387 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:17.641398 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:17.641450 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:17.641642 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:17.641643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:17.641790 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:17.641848 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.641850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.642542 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.642543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.643052 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.643054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.643437 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.643438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.643516 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.644022 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:17.644028 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:17.646006 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119}
W20260809 02:10:17.646649 14517 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:17.641286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:54659 (local address 127.5.9.190:36009)
0809 02:10:17.641353 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:17.641356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:17.641505 (+ 149us) server_negotiation.cc:407] Connection header received
0809 02:10:17.641535 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:17.641536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:17.641547 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:17.641617 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:17.641951 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.642455 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.643156 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.643359 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.643441 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.646098 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:54659: BlockingRecv error: recv got EOF from 127.5.9.129:54659 (error 108)
Metrics: {"server-negotiator.queue_time_us":47,"thread_start_us":120,"threads_started":1}
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:17.651309 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:17.646522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:53425)
0809 02:10:17.646761 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:17.646773 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:17.646834 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:17.647020 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:17.647021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:17.647160 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:17.647197 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.647199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.647874 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.647876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.648375 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.648376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.648763 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.648765 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.648819 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.649337 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:17.649342 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:17.651204 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219}
W20260809 02:10:17.651366 14519 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:17.646581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:53425 (local address 127.5.9.190:36009)
0809 02:10:17.646651 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:17.646653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:17.646885 (+ 232us) server_negotiation.cc:407] Connection header received
0809 02:10:17.646912 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:17.646913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:17.646923 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:17.647016 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:17.647277 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.647779 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.648463 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.648685 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.648766 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.651296 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:53425: BlockingRecv error: recv got EOF from 127.5.9.129:53425 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 02:10:17.651467 14514 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:10:17.729933 14518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:17.730237 14518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:17.730284 14518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:17.731583 14518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:17.731669 14518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:17.731714 14518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:10:17.733336 14518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.130.keytab
--principal=oryx/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:40755
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=41413
--tserver_master_addrs=127.5.9.190:36009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:17.733673 14518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:17.733948 14518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:17.739444 14526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:17.739461 14524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:17.739516 14518 server_base.cc:1034] running on GCE node
W20260809 02:10:17.739506 14528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:17.739869 14518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:17.740149 14518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:17.741405 14518 hybrid_clock.cc:648] HybridClock initialized: now 1786241417741392 us; error 49 us; skew 500 ppm
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241417, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:17.744279 14518 init.cc:377] Logged in from keytab as oryx/127.5.9.130@KRBTEST.COM (short username oryx)
I20260809 02:10:17.744822 14518 webserver.cc:460] Webserver started at http://127.5.9.130:41413/ using document root <none> and password file <none>
I20260809 02:10:17.745078 14518 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:17.745173 14518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:17.746857 14518 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:17.747633 14535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:17.747856 14518 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:17.747977 14518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "6fdf16f227f642318528ad8b83c3c476"
format_stamp: "Formatted at 2026-08-09 02:10:16 on dist-test-slave-p3t5"
server_key: "2a8fe90067f3e63c5ded00e79e33af1e"
server_key_iv: "e0fb7339630a989d94e787bd22c71726"
server_key_version: "encryptionkey@0"
I20260809 02:10:17.748368 14518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:17.767361 14518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:17.769526 14518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:17.769738 14518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:17.769964 14518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:17.770301 14518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:17.770336 14518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:17.770361 14518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:17.770416 14518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:17.776645 14518 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:40755
I20260809 02:10:17.776734 14658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:40755 every 8 connection(s)
I20260809 02:10:17.777129 14518 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 02:10:17.785367 14519 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:17.777994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:46025 (local address 127.5.9.190:36009)
0809 02:10:17.778047 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:17.778049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:17.778330 (+ 281us) server_negotiation.cc:407] Connection header received
0809 02:10:17.778902 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:17.778903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:17.778919 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:17.778982 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:17.780043 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.780567 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.781666 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.781874 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.781954 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.785309 (+ 3355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:46025: BlockingRecv error: recv got EOF from 127.5.9.130:46025 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:10:17.785441 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:17.777973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:46025)
0809 02:10:17.778155 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:17.778229 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:17.778774 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:17.779057 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:17.779062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:17.779702 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:17.779920 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.779926 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.780681 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.780682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.781530 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.781532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.781979 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.781980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.782111 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.782681 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:17.782689 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:17.785215 (+ 2526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":76,"threads_started":1}
W20260809 02:10:17.785853 14659 heartbeater.cc:644] Failed to heartbeat to 127.5.9.190:36009 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.9.190:36009: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
I20260809 02:10:17.786767 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 14518
I20260809 02:10:17.787079 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:45943
--local_ip_for_outbound_sockets=127.5.9.131
--tserver_master_addrs=127.5.9.190:36009
--webserver_port=41417
--webserver_interface=127.5.9.131
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.9.131
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.131.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:17.791216 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:17.786129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:47771)
0809 02:10:17.786192 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:17.786203 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:17.786249 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:17.786359 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:17.786360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:17.786484 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:17.786563 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.786565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.787291 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.787292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.787811 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.787813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.788296 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.788296 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.788346 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.788863 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:17.788871 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:17.791104 (+ 2233us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 02:10:17.791290 14519 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:17.786189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:47771 (local address 127.5.9.190:36009)
0809 02:10:17.786249 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:17.786251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:17.786268 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:10:17.786294 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:17.786295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:17.786304 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:17.786355 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:17.786636 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.787187 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.787899 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.788214 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.788293 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.791215 (+ 2922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:47771: BlockingRecv error: recv got EOF from 127.5.9.130:47771 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:17.796852 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:17.791617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:32789)
0809 02:10:17.791672 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:17.791683 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:17.791727 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:17.791891 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:17.791892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:17.792065 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:17.792110 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.792112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.792758 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.792760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.793737 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.793741 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.794143 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.794144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.794212 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.794641 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:17.794648 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:17.796740 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 02:10:17.796869 14519 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:17.791707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:32789 (local address 127.5.9.190:36009)
0809 02:10:17.791759 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:17.791761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:17.791777 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:10:17.791802 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:17.791803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:17.791812 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:17.791863 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:17.792192 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.792672 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.793834 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.794060 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.794121 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.796820 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:32789: BlockingRecv error: recv got EOF from 127.5.9.130:32789 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260809 02:10:17.797197 14659 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:10:17.874575 14662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:17.874800 14662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:17.874826 14662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:17.875963 14662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:17.875999 14662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:17.876025 14662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:10:17.877514 14662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/krb5kdc/oryx_127.5.9.131.keytab
--principal=oryx/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:45943
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=41417
--tserver_master_addrs=127.5.9.190:36009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:17.877848 14662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:17.878105 14662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:17.883790 14670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:17.883867 14668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:17.883970 14662 server_base.cc:1034] running on GCE node
W20260809 02:10:17.883877 14667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:17.884752 14662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:17.885041 14662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:17.886236 14662 hybrid_clock.cc:648] HybridClock initialized: now 1786241417886221 us; error 51 us; skew 500 ppm
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241417, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:17.889156 14662 init.cc:377] Logged in from keytab as oryx/127.5.9.131@KRBTEST.COM (short username oryx)
I20260809 02:10:17.889701 14662 webserver.cc:460] Webserver started at http://127.5.9.131:41417/ using document root <none> and password file <none>
I20260809 02:10:17.890138 14662 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:17.890283 14662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:17.892313 14662 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:10:17.893095 14678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:17.893304 14662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:10:17.893419 14662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "d39c9d5bc01f4923953b5c7de5c99c66"
format_stamp: "Formatted at 2026-08-09 02:10:17 on dist-test-slave-p3t5"
server_key: "049ef09f76ed9771775bc33eb1d37307"
server_key_iv: "889da130022437fd07e838f212409880"
server_key_version: "encryptionkey@0"
I20260809 02:10:17.893913 14662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:17.922914 14662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:17.925510 14662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:17.925724 14662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:17.926009 14662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:17.926410 14662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:17.926468 14662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:17.926517 14662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:17.926543 14662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:17.932653 14662 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:45943
I20260809 02:10:17.932701 14794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:45943 every 8 connection(s)
I20260809 02:10:17.933104 14662 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:10:17.934578 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 14662
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:17.943024 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:17.935170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37180 (local address 127.5.9.190:36009)
0809 02:10:17.935245 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:17.935247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:17.935310 (+ 63us) server_negotiation.cc:407] Connection header received
0809 02:10:17.935371 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:17.935372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:17.935385 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:17.935483 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:17.936165 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.936776 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.937718 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.938023 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.938110 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.939586 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:17.939591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:17.939591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:17.939603 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:17.941327 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:17.941758 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:17.941761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:17.941761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:17.941863 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:17.942090 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:17.942090 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:17.942091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:17.942440 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:17.942608 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:17.942744 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 02:10:17.942876 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 02:10:17.943174 14519 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:17.933943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:52101 (local address 127.5.9.190:36009)
0809 02:10:17.934019 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:17.934022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:17.934360 (+ 338us) server_negotiation.cc:407] Connection header received
0809 02:10:17.935070 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:17.935072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:17.935085 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:17.935160 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:17.936459 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.937072 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.938389 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.938573 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.938647 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.943093 (+ 4446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:52101: BlockingRecv error: recv got EOF from 127.5.9.131:52101 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:10:17.943207 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:17.933988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:52101)
0809 02:10:17.934211 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:17.934257 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:17.934945 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:17.935329 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:17.935334 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:17.936054 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:17.936346 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.936352 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.937175 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.937176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.938295 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.938297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.938686 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.938687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.938826 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.939358 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:17.939367 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:17.942985 (+ 3618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":103,"threads_started":1}
W20260809 02:10:17.943490 14795 heartbeater.cc:644] Failed to heartbeat to 127.5.9.190:36009 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.9.190:36009: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:17.948266 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:17.943770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:40559)
0809 02:10:17.943812 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:17.943821 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:17.943861 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:17.943999 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:17.943999 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:17.944155 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:17.944202 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.944204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.944848 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.944849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.945434 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.945435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.945882 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.945883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.945968 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.946374 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:17.946379 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:17.948212 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 02:10:17.948390 14517 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:17.943819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:40559 (local address 127.5.9.190:36009)
0809 02:10:17.943878 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:17.943880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:17.943887 (+ 7us) server_negotiation.cc:407] Connection header received
0809 02:10:17.943928 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:17.943929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:17.943938 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:17.943994 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:17.944282 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.944764 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.945517 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.945797 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.945871 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.948307 (+ 2436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:40559: BlockingRecv error: recv got EOF from 127.5.9.131:40559 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:17 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:17.952916 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:17.948596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:36879)
0809 02:10:17.948649 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:17.948661 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:17.948728 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:17.948848 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:17.948849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:17.948981 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:17.949016 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.949017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.949667 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.949668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.950213 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:17.950214 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.950582 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.950583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:17.950637 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.951089 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:17.951095 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:17.952823 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 02:10:17.952975 14517 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:17.948674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:36879 (local address 127.5.9.190:36009)
0809 02:10:17.948727 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:17.948730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:17.948739 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:10:17.948787 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:17.948788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:17.948797 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:17.948846 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:17.949091 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.949559 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.950301 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:17.950505 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:17.950584 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:17.952911 (+ 2327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:36879: BlockingRecv error: recv got EOF from 127.5.9.131:36879 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 02:10:17.953122 14795 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 02:10:18 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:18 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:18.656994 14799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:18.651925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:45939)
0809 02:10:18.652106 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:18.652121 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:18.652196 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:18.652443 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:18.652444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:18.652639 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:18.652693 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:18.652695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:18.653395 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:18.653396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:18.654038 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:18.654040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:18.654441 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:18.654442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:18.654492 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:18.654974 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:18.654978 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:18.656896 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
W20260809 02:10:18.657166 14800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:18.651998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:45939 (local address 127.5.9.190:36009)
0809 02:10:18.652172 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:18.652174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:18.652185 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:10:18.652262 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:18.652263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:18.652273 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:18.652338 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:18.652776 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:18.653298 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:18.654135 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:18.654355 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:18.654418 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:18.657082 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:45939: BlockingRecv error: recv got EOF from 127.5.9.129:45939 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":102,"threads_started":1}
Aug 09 02:10:18 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:18 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:18.803217 14801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:18.797759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:48973)
0809 02:10:18.797922 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:18.797934 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:18.798005 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:18.798164 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:18.798165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:18.798351 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:18.798406 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:18.798407 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:18.799155 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:18.799156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:18.799750 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:18.799752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:18.800181 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:18.800182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:18.800235 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:18.800779 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:18.800784 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:18.803144 (+ 2360us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
W20260809 02:10:18.803371 14800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:18.797876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:48973 (local address 127.5.9.190:36009)
0809 02:10:18.797943 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:18.797945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:18.797987 (+ 42us) server_negotiation.cc:407] Connection header received
0809 02:10:18.798064 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:18.798065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:18.798077 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:18.798143 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:18.798505 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:18.799048 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:18.799845 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:18.800063 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:18.800126 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:18.803287 (+ 3161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:48973: BlockingRecv error: recv got EOF from 127.5.9.130:48973 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 02:10:18 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:18 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:18.959033 14802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:18.953608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:60983)
0809 02:10:18.953897 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:18.953914 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:18.954024 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:18.954201 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:18.954202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:18.954406 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:18.954464 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:18.954466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:18.955171 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:18.955173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:18.955740 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:18.955743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:18.956145 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:18.956145 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:18.956207 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:18.956736 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:18.956741 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:18.958933 (+ 2192us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":110,"threads_started":1}
W20260809 02:10:18.959129 14800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:18.953782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:60983 (local address 127.5.9.190:36009)
0809 02:10:18.953902 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:18.953905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:18.954001 (+ 96us) server_negotiation.cc:407] Connection header received
0809 02:10:18.954100 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:18.954101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:18.954122 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:18.954192 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:18.954553 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:18.955074 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:18.955840 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:18.956053 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:18.956143 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:18.959060 (+ 2917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:60983: BlockingRecv error: recv got EOF from 127.5.9.131:60983 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 09 02:10:19 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:19 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:19.662911 14803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:19.657602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:47855)
0809 02:10:19.657853 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:19.657865 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:19.657924 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:19.658111 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:19.658112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:19.658304 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:19.658358 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:19.658360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:19.659073 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:19.659075 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:19.659639 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:19.659641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:19.659995 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:19.659996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:19.660053 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:19.660680 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:19.660685 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:19.662811 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":146,"threads_started":1}
W20260809 02:10:19.663038 14804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:19.657736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:47855 (local address 127.5.9.190:36009)
0809 02:10:19.657919 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:19.657921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:19.657933 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:10:19.657971 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:19.657972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:19.657983 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:19.658053 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:19.658443 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:19.658991 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:19.659719 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:19.659909 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:19.659961 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:19.662946 (+ 2985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:47855: BlockingRecv error: recv got EOF from 127.5.9.129:47855 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
Aug 09 02:10:19 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:19 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:19.809398 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:19.803792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:50011)
0809 02:10:19.803960 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:19.803970 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:19.804028 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:19.804181 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:19.804181 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:19.804362 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:19.804415 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:19.804417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:19.805147 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:19.805148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:19.805750 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:19.805752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:19.806259 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:19.806261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:19.806316 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:19.806950 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:19.806955 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:19.809324 (+ 2369us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":73,"threads_started":1}
W20260809 02:10:19.809528 14804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:19.803844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:50011 (local address 127.5.9.190:36009)
0809 02:10:19.803939 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:19.803941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:19.804043 (+ 102us) server_negotiation.cc:407] Connection header received
0809 02:10:19.804082 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:19.804083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:19.804096 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:19.804162 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:19.804484 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:19.805051 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:19.805855 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:19.806165 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:19.806254 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:19.809426 (+ 3172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:50011: BlockingRecv error: recv got EOF from 127.5.9.130:50011 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 02:10:19 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:19 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:19.965024 14806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:19.959634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:36851)
0809 02:10:19.959818 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:19.959855 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:19.959929 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:19.960099 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:19.960100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:19.960297 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:19.960348 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:19.960349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:19.961066 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:19.961067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:19.961770 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:19.961773 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:19.962191 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:19.962192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:19.962251 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:19.962809 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:19.962817 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:19.964933 (+ 2116us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":75,"threads_started":1}
W20260809 02:10:19.965133 14804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:19.959815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:36851 (local address 127.5.9.190:36009)
0809 02:10:19.959949 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:19.959952 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:19.959962 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:10:19.959992 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:19.959993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:19.960017 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:19.960090 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:19.960455 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:19.960974 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:19.961905 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:19.962097 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:19.962188 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:19.965053 (+ 2865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:36851: BlockingRecv error: recv got EOF from 127.5.9.131:36851 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 02:10:20 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:20 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:20.668604 14807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:20.663524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:36089)
0809 02:10:20.663705 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:20.663720 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:20.663790 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:20.663947 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:20.663948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:20.664147 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:20.664200 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:20.664201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:20.664867 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:20.664868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:20.665432 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:20.665434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:20.665868 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:20.665869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:20.665924 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:20.666505 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:20.666510 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:20.668522 (+ 2012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":80,"threads_started":1}
W20260809 02:10:20.668834 14808 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:20.663583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:36089 (local address 127.5.9.190:36009)
0809 02:10:20.663755 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:20.663757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:20.663771 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:10:20.663850 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:20.663851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:20.663862 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:20.663928 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:20.664267 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:20.664779 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:20.665541 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:20.665764 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:20.665853 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:20.668714 (+ 2861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:36089: BlockingRecv error: recv got EOF from 127.5.9.129:36089 (error 108)
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":107,"threads_started":1}
Aug 09 02:10:20 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:20 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:20.815166 14809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:20.809929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:52483)
0809 02:10:20.810162 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:20.810176 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:20.810276 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:20.810486 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:20.810487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:20.810709 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:20.810758 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:20.810761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:20.811488 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:20.811489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:20.812027 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:20.812028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:20.812417 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:20.812418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:20.812479 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:20.813025 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:20.813030 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:20.815061 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":116,"threads_started":1}
W20260809 02:10:20.815219 14808 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:20.810043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:52483 (local address 127.5.9.190:36009)
0809 02:10:20.810143 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:20.810146 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:20.810255 (+ 109us) server_negotiation.cc:407] Connection header received
0809 02:10:20.810373 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:20.810374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:20.810393 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:20.810470 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:20.810845 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:20.811398 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:20.812123 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:20.812338 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:20.812417 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:20.815160 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:52483: BlockingRecv error: recv got EOF from 127.5.9.130:52483 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 02:10:20 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:20 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:20.970623 14810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:20.965751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:52537)
0809 02:10:20.965941 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:20.965951 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:20.966004 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:20.966153 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:20.966154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:20.966331 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:20.966395 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:20.966396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:20.967066 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:20.967067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:20.967611 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:20.967612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:20.967983 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:20.967984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:20.968029 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:20.968501 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:20.968506 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:20.970548 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":95,"threads_started":1}
W20260809 02:10:20.970767 14808 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:20.965828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:52537 (local address 127.5.9.190:36009)
0809 02:10:20.965933 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:20.965935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:20.966009 (+ 74us) server_negotiation.cc:407] Connection header received
0809 02:10:20.966054 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:20.966055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:20.966068 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:20.966133 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:20.966460 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:20.966990 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:20.967703 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:20.967894 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:20.967974 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:20.970675 (+ 2701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:52537: BlockingRecv error: recv got EOF from 127.5.9.131:52537 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 02:10:21 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:21 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:21.676220 14812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:21.669237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:44445)
0809 02:10:21.669417 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:21.669430 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:21.669499 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:21.669763 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:21.669765 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:21.670197 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:21.670272 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:21.670276 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:21.671494 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:21.671496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:21.672484 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:21.672488 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:21.673037 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:21.673039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:21.673112 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:21.673801 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:21.673808 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:21.676135 (+ 2327us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
W20260809 02:10:21.676350 14813 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:21.669343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:44445 (local address 127.5.9.190:36009)
0809 02:10:21.669522 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:21.669524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:21.669534 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:10:21.669564 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:21.669565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:21.669576 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:21.669694 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:21.670411 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:21.671379 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:21.672581 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:21.672952 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:21.673024 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:21.676267 (+ 3243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:44445: BlockingRecv error: recv got EOF from 127.5.9.129:44445 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":116,"threads_started":1}
Aug 09 02:10:21 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:21 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:21.821403 14814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:21.815911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:45291)
0809 02:10:21.816066 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:21.816080 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:21.816145 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:21.816350 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:21.816352 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:21.816664 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:21.816732 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:21.816735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:21.817522 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:21.817523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:21.818167 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:21.818169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:21.818604 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:21.818606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:21.818671 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:21.819292 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:21.819299 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:21.821334 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":97,"threads_started":1}
W20260809 02:10:21.821553 14813 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:21.815995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:45291 (local address 127.5.9.190:36009)
0809 02:10:21.816157 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:21.816159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:21.816186 (+ 27us) server_negotiation.cc:407] Connection header received
0809 02:10:21.816212 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:21.816213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:21.816230 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:21.816310 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:21.816833 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:21.817404 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:21.818274 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:21.818512 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:21.818606 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:21.821461 (+ 2855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:45291: BlockingRecv error: recv got EOF from 127.5.9.130:45291 (error 108)
Metrics: {"server-negotiator.queue_time_us":136}
Aug 09 02:10:21 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:21 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:21.977344 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:21.971167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:33661)
0809 02:10:21.971375 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:21.971390 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:21.971450 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:21.971639 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:21.971639 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:21.971830 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:21.971882 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:21.971884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:21.972996 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:21.972997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:21.973824 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:21.973828 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:21.974430 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:21.974431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:21.974475 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:21.975050 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:21.975055 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:21.977278 (+ 2223us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":111,"threads_started":1}
W20260809 02:10:21.977523 14813 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:21.971304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:33661 (local address 127.5.9.190:36009)
0809 02:10:21.971395 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:21.971398 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:21.971440 (+ 42us) server_negotiation.cc:407] Connection header received
0809 02:10:21.971512 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:21.971513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:21.971533 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:21.971626 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:21.971968 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:21.972895 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:21.973957 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:21.974338 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:21.974434 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:21.977427 (+ 2993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:33661: BlockingRecv error: recv got EOF from 127.5.9.131:33661 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 02:10:22 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:22 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:22.681974 14816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:22.676926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:34413)
0809 02:10:22.677084 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:22.677097 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:22.677157 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:22.677319 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:22.677320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:22.677505 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:22.677569 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:22.677571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:22.678330 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:22.678331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:22.678896 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:22.678898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:22.679258 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:22.679259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:22.679300 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:22.679796 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:22.679801 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:22.681867 (+ 2066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
W20260809 02:10:22.682060 14817 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:22.676983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:34413 (local address 127.5.9.190:36009)
0809 02:10:22.677143 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:22.677145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:22.677155 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:10:22.677217 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:22.677218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:22.677228 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:22.677296 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:22.677713 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:22.678254 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:22.678990 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:22.679188 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:22.679245 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:22.681983 (+ 2738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:34413: BlockingRecv error: recv got EOF from 127.5.9.129:34413 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
Aug 09 02:10:22 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:22 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:22.827201 14818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:22.821977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:37035)
0809 02:10:22.822149 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:22.822182 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:22.822247 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:22.822422 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:22.822422 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:22.822620 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:22.822670 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:22.822672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:22.823394 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:22.823395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:22.823945 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:22.823946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:22.824335 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:22.824336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:22.824444 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:22.824954 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:22.824959 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:22.827119 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
W20260809 02:10:22.827366 14817 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:22.822140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:37035 (local address 127.5.9.190:36009)
0809 02:10:22.822222 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:22.822224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:22.822233 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:10:22.822325 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:22.822326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:22.822346 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:22.822411 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:22.822757 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:22.823292 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:22.824037 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:22.824241 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:22.824323 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:22.827258 (+ 2935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:37035: BlockingRecv error: recv got EOF from 127.5.9.130:37035 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 02:10:22 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:22 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:22.983206 14819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:22.978161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:35023)
0809 02:10:22.978306 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:22.978318 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:22.978394 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:22.978537 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:22.978538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:22.978718 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:22.978769 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:22.978770 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:22.979503 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:22.979504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:22.980040 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:22.980042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:22.980401 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:22.980402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:22.980446 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:22.980982 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:22.980988 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:22.983139 (+ 2151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
W20260809 02:10:22.983364 14817 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:22.978237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:35023 (local address 127.5.9.190:36009)
0809 02:10:22.978327 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:22.978330 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:22.978395 (+ 65us) server_negotiation.cc:407] Connection header received
0809 02:10:22.978456 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:22.978457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:22.978471 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:22.978518 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:22.978882 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:22.979404 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:22.980131 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:22.980327 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:22.980388 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:22.983274 (+ 2886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:35023: BlockingRecv error: recv got EOF from 127.5.9.131:35023 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 02:10:23 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:23 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:23.687954 14820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:23.682672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:37685)
0809 02:10:23.682858 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:23.682869 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:23.682926 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:23.683093 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:23.683093 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:23.683282 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:23.683339 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:23.683340 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:23.684049 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:23.684050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:23.684691 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:23.684693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:23.685211 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:23.685212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:23.685261 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:23.685868 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:23.685873 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:23.687840 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":81,"threads_started":1}
W20260809 02:10:23.688042 14821 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:23.682741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:37685 (local address 127.5.9.190:36009)
0809 02:10:23.682904 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:23.682906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:23.682918 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:10:23.683004 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:23.683004 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:23.683016 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:23.683072 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:23.683423 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:23.683951 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:23.684786 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:23.685126 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:23.685195 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:23.687966 (+ 2771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:37685: BlockingRecv error: recv got EOF from 127.5.9.129:37685 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
Aug 09 02:10:23 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:23 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:23.833045 14822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:23.827877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:54539)
0809 02:10:23.828102 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:23.828113 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:23.828203 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:23.828424 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:23.828425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:23.828686 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:23.828743 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:23.828745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:23.829462 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:23.829463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:23.830144 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:23.830147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:23.830503 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:23.830504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:23.830547 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:23.831058 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:23.831063 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:23.832979 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":102,"threads_started":1}
W20260809 02:10:23.833199 14821 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:23.827940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:54539 (local address 127.5.9.190:36009)
0809 02:10:23.828025 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:23.828027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:23.828186 (+ 159us) server_negotiation.cc:407] Connection header received
0809 02:10:23.828318 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:23.828320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:23.828336 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:23.828392 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:23.828838 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:23.829366 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:23.830229 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:23.830421 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:23.830493 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:23.833111 (+ 2618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:54539: BlockingRecv error: recv got EOF from 127.5.9.130:54539 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 02:10:23 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:23 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:23.988965 14823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:23.983770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:51479)
0809 02:10:23.983984 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:23.983999 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:23.984082 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:23.984294 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:23.984295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:23.984553 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:23.984624 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:23.984627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:23.985317 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:23.985318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:23.986005 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:23.986008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:23.986350 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:23.986351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:23.986393 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:23.986991 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:23.986996 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:23.988901 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":123,"threads_started":1}
W20260809 02:10:23.989143 14821 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:23.983908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:51479 (local address 127.5.9.190:36009)
0809 02:10:23.983988 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:23.983991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:23.984077 (+ 86us) server_negotiation.cc:407] Connection header received
0809 02:10:23.984155 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:23.984156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:23.984172 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:23.984232 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:23.984704 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:23.985233 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:23.986091 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:23.986280 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:23.986338 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:23.989037 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:51479: BlockingRecv error: recv got EOF from 127.5.9.131:51479 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 02:10:24 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:24 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:24.693653 14824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:24.688553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:37395)
0809 02:10:24.688705 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:24.688715 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:24.688766 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:24.688961 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:24.688962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:24.689134 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:24.689186 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:24.689187 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:24.689932 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:24.689933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:24.690612 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:24.690616 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:24.691062 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:24.691063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:24.691129 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:24.691611 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:24.691616 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:24.693531 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":67,"threads_started":1}
W20260809 02:10:24.693737 14825 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:24.688646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:37395 (local address 127.5.9.190:36009)
0809 02:10:24.688804 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:24.688806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:24.688815 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:10:24.688844 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:24.688845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:24.688856 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:24.688941 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:24.689263 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:24.689852 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:24.690709 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:24.690978 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:24.691064 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:24.693658 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:37395: BlockingRecv error: recv got EOF from 127.5.9.129:37395 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
Aug 09 02:10:24 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:24 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:24.838835 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:24.833752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:42137)
0809 02:10:24.833960 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:24.833975 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:24.834061 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:24.834235 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:24.834237 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:24.834479 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:24.834529 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:24.834531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:24.835239 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:24.835241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:24.835801 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:24.835803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:24.836190 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:24.836191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:24.836252 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:24.836749 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:24.836754 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:24.838769 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":133,"threads_started":1}
W20260809 02:10:24.839028 14825 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:24.833942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:42137 (local address 127.5.9.190:36009)
0809 02:10:24.834016 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:24.834018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:24.834035 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:10:24.834128 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:24.834129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:24.834143 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:24.834226 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:24.834623 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:24.835143 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:24.835898 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:24.836109 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:24.836191 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:24.838956 (+ 2765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:42137: BlockingRecv error: recv got EOF from 127.5.9.130:42137 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 02:10:24 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:24 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:24.994750 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:24.989510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:53579)
0809 02:10:24.989737 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:24.989751 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:24.989859 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:24.990029 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:24.990030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:24.990235 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:24.990286 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:24.990288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:24.991001 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:24.991002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:24.991546 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:24.991548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:24.991956 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:24.991957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:24.992019 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:24.992558 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:24.992563 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:24.994649 (+ 2086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":136,"threads_started":1}
W20260809 02:10:24.994809 14825 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:24.989644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:53579 (local address 127.5.9.190:36009)
0809 02:10:24.989724 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:24.989727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:24.989839 (+ 112us) server_negotiation.cc:407] Connection header received
0809 02:10:24.989922 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:24.989923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:24.989938 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:24.990019 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:24.990381 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:24.990909 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:24.991642 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:24.991837 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:24.991956 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:24.994739 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:53579: BlockingRecv error: recv got EOF from 127.5.9.131:53579 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 02:10:25 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:25 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:25.700090 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:25.694311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:43267)
0809 02:10:25.694513 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:25.694529 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:25.694614 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:25.694923 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:25.694924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:25.695112 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:25.695165 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:25.695167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:25.695963 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:25.695965 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:25.696979 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:25.696983 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:25.697402 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:25.697402 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:25.697468 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:25.698071 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:25.698077 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:25.700015 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":88,"threads_started":1}
W20260809 02:10:25.700296 14830 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:25.694508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:43267 (local address 127.5.9.190:36009)
0809 02:10:25.694714 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:25.694716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:25.694726 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:10:25.694756 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:25.694757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:25.694768 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:25.694878 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:25.695267 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:25.695860 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:25.697060 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:25.697296 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:25.697360 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:25.700192 (+ 2832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:43267: BlockingRecv error: recv got EOF from 127.5.9.129:43267 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":90,"threads_started":1}
Aug 09 02:10:25 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:25 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:25.845391 14831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:25.839445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:44047)
0809 02:10:25.839730 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:25.839745 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:25.839838 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:25.840062 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:25.840063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:25.840308 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:25.840364 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:25.840366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:25.841329 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:25.841331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:25.841994 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:25.841996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:25.842427 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:25.842428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:25.842493 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:25.843114 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:25.843120 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:25.845324 (+ 2204us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":126,"threads_started":1}
W20260809 02:10:25.845510 14830 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:25.839591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:44047 (local address 127.5.9.190:36009)
0809 02:10:25.839716 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:25.839719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:25.839812 (+ 93us) server_negotiation.cc:407] Connection header received
0809 02:10:25.839946 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:25.839947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:25.839966 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:25.840052 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:25.840491 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:25.841218 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:25.842100 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:25.842335 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:25.842428 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:25.845431 (+ 3003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:44047: BlockingRecv error: recv got EOF from 127.5.9.130:44047 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 02:10:25 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:25 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:26.000662 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:25.995387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:48599)
0809 02:10:25.995606 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:25.995624 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:25.995715 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:25.995878 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:25.995879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:25.996071 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:25.996121 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:25.996123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:25.996944 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:25.996946 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:25.997484 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:25.997485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:25.997949 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:25.997950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:25.998011 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:25.998484 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:25.998489 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:26.000595 (+ 2106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":149,"threads_started":1}
W20260809 02:10:26.000842 14830 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:25.995556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:48599 (local address 127.5.9.190:36009)
0809 02:10:25.995633 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:25.995635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:25.995706 (+ 71us) server_negotiation.cc:407] Connection header received
0809 02:10:25.995789 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:25.995790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:25.995805 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:25.995866 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:25.996214 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:25.996806 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:25.997578 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:25.997874 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:25.997951 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:26.000758 (+ 2807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:48599: BlockingRecv error: recv got EOF from 127.5.9.131:48599 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 02:10:26 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:26 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:26.712273 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:26.700585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:51525)
0809 02:10:26.700812 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:26.700854 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:26.700937 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:26.701108 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:26.701109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:26.701320 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:26.701372 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:26.701374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:26.702200 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:26.702201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:26.702739 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:26.702741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:26.703134 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:26.703135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:26.703205 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:26.710088 (+ 6883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:26.710096 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:26.712193 (+ 2097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":108,"threads_started":1}
W20260809 02:10:26.712400 14835 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:26.700685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:51525 (local address 127.5.9.190:36009)
0809 02:10:26.700814 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:26.700816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:26.700935 (+ 119us) server_negotiation.cc:407] Connection header received
0809 02:10:26.701000 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:26.701002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:26.701020 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:26.701098 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:26.701463 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:26.702106 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:26.702833 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:26.703043 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:26.703132 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:26.712312 (+ 9180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:51525: BlockingRecv error: recv got EOF from 127.5.9.129:51525 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
Aug 09 02:10:26 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:26 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:26.851352 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:26.845964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:34863)
0809 02:10:26.846290 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:26.846303 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:26.846391 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:26.846554 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:26.846555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:26.846786 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:26.846864 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:26.846867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:26.847576 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:26.847577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:26.848141 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:26.848143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:26.848543 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:26.848544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:26.848606 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:26.849158 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:26.849164 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:26.851251 (+ 2087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":114,"threads_started":1}
W20260809 02:10:26.851428 14835 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:26.846082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:34863 (local address 127.5.9.190:36009)
0809 02:10:26.846271 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:26.846274 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:26.846367 (+ 93us) server_negotiation.cc:407] Connection header received
0809 02:10:26.846453 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:26.846454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:26.846469 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:26.846544 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:26.846961 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:26.847482 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:26.848237 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:26.848450 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:26.848542 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:26.851360 (+ 2818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:34863: BlockingRecv error: recv got EOF from 127.5.9.130:34863 (error 108)
Metrics: {"server-negotiator.queue_time_us":155}
Aug 09 02:10:27 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:27 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:27.006106 14837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:27.001151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:52961)
0809 02:10:27.001314 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:27.001326 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:27.001413 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:27.001572 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:27.001573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:27.001849 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:27.001900 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:27.001902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:27.002554 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:27.002555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:27.003089 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:27.003091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:27.003441 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:27.003441 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:27.003482 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:27.003939 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:27.003943 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:27.006044 (+ 2101us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":91,"threads_started":1}
W20260809 02:10:27.006242 14835 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:27.001219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:52961 (local address 127.5.9.190:36009)
0809 02:10:27.001291 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:27.001294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:27.001405 (+ 111us) server_negotiation.cc:407] Connection header received
0809 02:10:27.001484 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:27.001485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:27.001508 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:27.001549 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:27.001979 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:27.002485 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:27.003180 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:27.003370 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:27.003430 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:27.006155 (+ 2725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:52961: BlockingRecv error: recv got EOF from 127.5.9.131:52961 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 02:10:27 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:27 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:27.717536 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:27.712838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:42111)
0809 02:10:27.713018 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:27.713030 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:27.713083 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:27.713224 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:27.713224 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:27.713404 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:27.713460 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:27.713462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:27.714179 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:27.714180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:27.714714 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:27.714716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:27.715069 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:27.715070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:27.715111 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:27.715625 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:27.715630 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:27.717465 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":92,"threads_started":1}
W20260809 02:10:27.717713 14839 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:27.712939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:42111 (local address 127.5.9.190:36009)
0809 02:10:27.713090 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:27.713092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:27.713102 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:10:27.713128 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:27.713129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:27.713139 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:27.713205 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:27.713535 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:27.714103 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:27.714796 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:27.715003 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:27.715057 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:27.717587 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:42111: BlockingRecv error: recv got EOF from 127.5.9.129:42111 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
Aug 09 02:10:27 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:27 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:27.857503 14840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:27.852014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:39525)
0809 02:10:27.852197 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:27.852209 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:27.852289 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:27.852493 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:27.852494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:27.852684 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:27.852738 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:27.852739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:27.853500 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:27.853501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:27.854075 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:27.854077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:27.854498 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:27.854499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:27.854545 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:27.855168 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:27.855180 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:27.857420 (+ 2240us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":128,"threads_started":1}
W20260809 02:10:27.857678 14839 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:27.852124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:39525 (local address 127.5.9.190:36009)
0809 02:10:27.852200 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:27.852202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:27.852326 (+ 124us) server_negotiation.cc:407] Connection header received
0809 02:10:27.852387 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:27.852388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:27.852402 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:27.852470 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:27.852845 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:27.853390 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:27.854181 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:27.854389 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:27.854458 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:27.857567 (+ 3109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:39525: BlockingRecv error: recv got EOF from 127.5.9.130:39525 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 02:10:28 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:28 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:28.012513 14841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:28.006769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:60137)
0809 02:10:28.006903 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:28.006916 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:28.006974 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:28.007394 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:28.007396 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:28.007638 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:28.007714 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:28.007717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:28.008460 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:28.008461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:28.009228 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:28.009231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:28.009830 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:28.009831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:28.009909 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:28.010533 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:28.010538 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:28.012446 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
W20260809 02:10:28.012701 14839 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:28.006895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:60137 (local address 127.5.9.190:36009)
0809 02:10:28.007063 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:28.007065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:28.007076 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:10:28.007150 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:28.007151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:28.007167 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:28.007329 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:28.007844 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:28.008369 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:28.009350 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:28.009729 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:28.009807 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:28.012624 (+ 2817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:60137: BlockingRecv error: recv got EOF from 127.5.9.131:60137 (error 108)
Metrics: {"server-negotiator.queue_time_us":142}
Aug 09 02:10:28 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:28 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:28.723546 14842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:28.718162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:54501)
0809 02:10:28.718350 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:28.718364 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:28.718443 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:28.718654 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:28.718655 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:28.718863 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:28.718926 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:28.718927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:28.719616 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:28.719617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:28.720197 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:28.720198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:28.720661 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:28.720662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:28.720765 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:28.721374 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:28.721379 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:28.723470 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":109,"threads_started":1}
W20260809 02:10:28.723718 14843 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:28.718293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:54501 (local address 127.5.9.190:36009)
0809 02:10:28.718439 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:28.718441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:28.718451 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:10:28.718505 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:28.718506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:28.718517 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:28.718584 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:28.719009 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:28.719524 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:28.720294 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:28.720574 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:28.720664 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:28.723619 (+ 2955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:54501: BlockingRecv error: recv got EOF from 127.5.9.129:54501 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
Aug 09 02:10:28 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:28 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:28.863521 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:28.858188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:55413)
0809 02:10:28.858385 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:28.858398 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:28.858476 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:28.858618 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:28.858619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:28.858802 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:28.858854 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:28.858856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:28.859661 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:28.859662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:28.860194 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:28.860196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:28.860548 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:28.860548 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:28.860589 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:28.861153 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:28.861158 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:28.863458 (+ 2300us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":116,"threads_started":1}
W20260809 02:10:28.863679 14843 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:28.858271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:55413 (local address 127.5.9.190:36009)
0809 02:10:28.858329 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:28.858332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:28.858473 (+ 141us) server_negotiation.cc:407] Connection header received
0809 02:10:28.858528 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:28.858529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:28.858542 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:28.858594 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:28.858925 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:28.859576 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:28.860286 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:28.860479 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:28.860534 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:28.863599 (+ 3065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:55413: BlockingRecv error: recv got EOF from 127.5.9.130:55413 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 02:10:29 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:29 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:29.019253 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:29.013160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:44809)
0809 02:10:29.013311 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:29.013322 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:29.013391 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:29.013553 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:29.013554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:29.013799 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:29.013850 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:29.013852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:29.014609 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:29.014610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:29.015590 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:29.015594 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:29.016057 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:29.016058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:29.016120 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:29.016691 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:29.016697 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:29.019187 (+ 2490us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":68,"threads_started":1}
W20260809 02:10:29.019379 14843 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:29.013227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:44809 (local address 127.5.9.190:36009)
0809 02:10:29.013283 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:29.013286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:29.013371 (+ 85us) server_negotiation.cc:407] Connection header received
0809 02:10:29.013453 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:29.013454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:29.013489 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:29.013546 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:29.013943 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:29.014527 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:29.015683 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:29.015961 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:29.016053 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:29.019281 (+ 3228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:44809: BlockingRecv error: recv got EOF from 127.5.9.131:44809 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 02:10:29 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:29 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:29.729800 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:29.724280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:39885)
0809 02:10:29.724439 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:29.724459 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:29.724514 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:29.724697 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:29.724698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:29.724883 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:29.724935 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:29.724937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:29.725608 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:29.725609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:29.726238 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:29.726240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:29.726794 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:29.726796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:29.726901 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:29.727561 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:29.727567 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:29.729725 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
W20260809 02:10:29.729934 14847 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:29.724387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:39885 (local address 127.5.9.190:36009)
0809 02:10:29.724547 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:29.724555 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:29.724565 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:10:29.724594 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:29.724595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:29.724605 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:29.724678 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:29.725008 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:29.725518 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:29.726327 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:29.726644 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:29.726721 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:29.729833 (+ 3112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:39885: BlockingRecv error: recv got EOF from 127.5.9.129:39885 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
Aug 09 02:10:29 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:29 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:29.868824 14848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:29.864054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:43683)
0809 02:10:29.864197 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:29.864210 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:29.864286 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:29.864449 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:29.864450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:29.864635 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:29.864687 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:29.864689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:29.865370 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:29.865371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:29.865950 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:29.865952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:29.866308 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:29.866309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:29.866354 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:29.866852 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:29.866856 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:29.868748 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
W20260809 02:10:29.868999 14847 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:29.864166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:43683 (local address 127.5.9.190:36009)
0809 02:10:29.864228 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:29.864230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:29.864269 (+ 39us) server_negotiation.cc:407] Connection header received
0809 02:10:29.864349 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:29.864349 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:29.864362 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:29.864424 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:29.864761 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:29.865277 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:29.866036 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:29.866234 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:29.866293 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:29.868929 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:43683: BlockingRecv error: recv got EOF from 127.5.9.130:43683 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 02:10:30 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:30 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:30.024935 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:30.019788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:47275)
0809 02:10:30.020011 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:30.020025 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:30.020117 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:30.020259 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:30.020260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:30.020465 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:30.020516 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:30.020518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:30.021250 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:30.021251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:30.021818 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:30.021820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:30.022202 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:30.022203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:30.022267 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:30.022811 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:30.022816 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:30.024803 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":126,"threads_started":1}
W20260809 02:10:30.024989 14847 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:30.019915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:47275 (local address 127.5.9.190:36009)
0809 02:10:30.019994 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:30.019996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:30.020099 (+ 103us) server_negotiation.cc:407] Connection header received
0809 02:10:30.020177 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:30.020178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:30.020195 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:30.020261 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:30.020602 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:30.021135 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:30.021914 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:30.022120 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:30.022202 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:30.024917 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:47275: BlockingRecv error: recv got EOF from 127.5.9.131:47275 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 02:10:30 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:30 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:30.735786 14850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:30.730524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:33523)
0809 02:10:30.730720 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:30.730735 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:30.730818 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:30.731035 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:30.731036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:30.731217 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:30.731286 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:30.731288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:30.732122 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:30.732123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:30.732697 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:30.732698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:30.733093 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:30.733094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:30.733144 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:30.733741 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:30.733746 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:30.735711 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":83,"threads_started":1}
W20260809 02:10:30.736011 14851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:30.730652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:33523 (local address 127.5.9.190:36009)
0809 02:10:30.730855 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:30.730858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:30.730868 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:10:30.730903 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:30.730904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:30.730915 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:30.730982 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:30.731411 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:30.732026 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:30.732796 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:30.733016 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:30.733076 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:30.735907 (+ 2831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:33523: BlockingRecv error: recv got EOF from 127.5.9.129:33523 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":100,"threads_started":1}
Aug 09 02:10:30 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:30 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:30.874200 14852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:30.869373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:57117)
0809 02:10:30.869520 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:30.869534 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:30.869647 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:30.869839 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:30.869840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:30.870032 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:30.870087 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:30.870088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:30.870749 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:30.870749 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:30.871294 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:30.871296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:30.871654 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:30.871655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:30.871696 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:30.872196 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:30.872201 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:30.874135 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
W20260809 02:10:30.874315 14851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:30.869483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:57117 (local address 127.5.9.190:36009)
0809 02:10:30.869532 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:30.869535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:30.869635 (+ 100us) server_negotiation.cc:407] Connection header received
0809 02:10:30.869720 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:30.869721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:30.869735 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:30.869828 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:30.870158 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:30.870677 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:30.871379 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:30.871583 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:30.871642 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:30.874240 (+ 2598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:57117: BlockingRecv error: recv got EOF from 127.5.9.130:57117 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 02:10:31 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:31 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:31.030517 14853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:31.025535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:47965)
0809 02:10:31.025721 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:31.025734 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:31.025801 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:31.025948 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:31.025948 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:31.026124 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:31.026176 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:31.026177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:31.026916 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:31.026917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:31.027448 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:31.027450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:31.027879 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:31.027880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:31.027933 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:31.028447 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:31.028452 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:31.030432 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":132,"threads_started":1}
W20260809 02:10:31.030721 14851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:31.025689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:47965 (local address 127.5.9.190:36009)
0809 02:10:31.025801 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:31.025804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:31.025839 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:10:31.025863 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:31.025863 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:31.025877 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:31.025920 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:31.026275 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:31.026829 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:31.027541 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:31.027792 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:31.027854 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:31.030582 (+ 2728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:47965: BlockingRecv error: recv got EOF from 127.5.9.131:47965 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 02:10:31 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:31 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:31.742857 14855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:31.736329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:54263)
0809 02:10:31.736481 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:31.736494 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:31.736554 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:31.736753 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:31.736754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:31.736931 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:31.736984 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:31.736986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:31.737993 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:31.737994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:31.738911 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:31.738931 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:31.739400 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:31.739401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:31.739475 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:31.740135 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:31.740142 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:31.742736 (+ 2594us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":66,"threads_started":1}
W20260809 02:10:31.742980 14856 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:31.736398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:54263 (local address 127.5.9.190:36009)
0809 02:10:31.736549 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:31.736556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:31.736566 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:10:31.736629 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:31.736630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:31.736645 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:31.736743 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:31.737068 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:31.737917 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:31.739011 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:31.739293 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:31.739388 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:31.742896 (+ 3508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:54263: BlockingRecv error: recv got EOF from 127.5.9.129:54263 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
Aug 09 02:10:31 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:31 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:31.879920 14857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:31.874855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:54043)
0809 02:10:31.875038 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:31.875050 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:31.875122 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:31.875282 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:31.875283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:31.875499 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:31.875549 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:31.875551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:31.876299 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:31.876301 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:31.876847 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:31.876848 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:31.877248 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:31.877249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:31.877310 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:31.877931 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:31.877936 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:31.879800 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":92,"threads_started":1}
W20260809 02:10:31.879976 14856 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:31.874938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:54043 (local address 127.5.9.190:36009)
0809 02:10:31.875038 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:31.875040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:31.875101 (+ 61us) server_negotiation.cc:407] Connection header received
0809 02:10:31.875181 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:31.875182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:31.875196 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:31.875276 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:31.875655 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:31.876191 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:31.876945 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:31.877155 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:31.877247 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:31.879908 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:54043: BlockingRecv error: recv got EOF from 127.5.9.130:54043 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 02:10:32 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:32 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.131@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:32.035933 14858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:32.031067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.131:33675)
0809 02:10:32.031226 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:32.031242 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:32.031317 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:32.031464 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:32.031465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:32.031647 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:32.031697 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:32.031699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:32.032450 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:32.032452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:32.032984 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:32.032986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:32.033373 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:32.033374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:32.033426 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:32.033915 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:32.033920 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:32.035811 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
W20260809 02:10:32.035982 14856 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:32.031191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:33675 (local address 127.5.9.190:36009)
0809 02:10:32.031262 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:32.031264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:32.031299 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:10:32.031374 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:32.031375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:32.031388 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:32.031433 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:32.031814 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:32.032340 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:32.033089 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:32.033281 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:32.033342 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:32.035909 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.131:33675: BlockingRecv error: recv got EOF from 127.5.9.131:33675 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 02:10:32 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:32 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.129@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:32.748754 14859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:32.743549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.129:36817)
0809 02:10:32.743707 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:32.743719 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:32.743808 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:32.743945 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:32.743945 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:32.744147 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:32.744201 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:32.744203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:32.744973 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:32.744975 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:32.745544 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:32.745547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:32.745999 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:32.746001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:32.746052 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:32.746562 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:32.746567 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:32.748640 (+ 2073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
W20260809 02:10:32.748960 14860 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:32.743667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:36817 (local address 127.5.9.190:36009)
0809 02:10:32.743810 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:32.743812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:32.743849 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:10:32.743874 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:32.743875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:32.743886 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:32.743939 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:32.744295 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:32.744867 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:32.745695 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:32.745896 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:32.745964 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:32.748745 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.129:36817: BlockingRecv error: recv got EOF from 127.5.9.129:36817 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
Aug 09 02:10:32 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:10:32 dist-test-slave-p3t5 krb5kdc[13740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.9.130@KRBTEST.COM for oryx/127.5.9.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:10:32.885912 14861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:32.880606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:36009 (local address 127.5.9.130:58721)
0809 02:10:32.880926 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:32.880939 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:32.881053 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:32.881292 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:32.881293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:32.881554 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:32.881719 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:32.881722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:32.882511 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:32.882512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:32.883052 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:32.883053 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:32.883385 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:32.883385 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:32.883427 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:32.883912 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:32.883916 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:32.885834 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.9.190:36009: Server oryx/127.5.9.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":110,"threads_started":1}
W20260809 02:10:32.886041 14860 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:32.880718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:58721 (local address 127.5.9.190:36009)
0809 02:10:32.880919 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:32.880922 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:32.881053 (+ 131us) server_negotiation.cc:407] Connection header received
0809 02:10:32.881176 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:32.881178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:32.881196 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:32.881268 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:32.881860 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:32.882418 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:32.883129 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:32.883316 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:32.883367 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:32.885938 (+ 2571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.9.130:58721: BlockingRecv error: recv got EOF from 127.5.9.130:58721 (error 108)
Metrics: {"server-negotiator.queue_time_us":164}
I20260809 02:10:32.935746 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 14361
I20260809 02:10:32.942713 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 14518
I20260809 02:10:32.949397 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 14662
I20260809 02:10:32.955188 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 14287
2026-08-09T02:10:32Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17700 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T02:10:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:10:32Z Disabled control of system clock
I20260809 02:10:32.986914 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:34519
--webserver_interface=127.5.9.190
--webserver_port=0
--builtin_ntp_servers=127.5.9.148:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:34519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 02:10:33.073899 14865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:33.074139 14865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:33.074162 14865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:33.075350 14865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:33.075388 14865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:33.075400 14865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:33.075413 14865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:33.075425 14865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:10:33.076789 14865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:34519
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:34519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:33.077023 14865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:33.077304 14865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:33.082472 14871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:33.082484 14870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:33.082469 14874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:33.082741 14865 server_base.cc:1034] running on GCE node
I20260809 02:10:33.083056 14865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:33.083384 14865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:33.084565 14865 hybrid_clock.cc:648] HybridClock initialized: now 1786241433084560 us; error 39 us; skew 500 ppm
I20260809 02:10:33.085995 14865 webserver.cc:460] Webserver started at http://127.5.9.190:42917/ using document root <none> and password file <none>
I20260809 02:10:33.086282 14865 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:33.086359 14865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:33.086521 14865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:33.087725 14865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "f2d57782ea3f4fc2a043a4faa581572f"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "69e78f544373ab93b40d037de150299d"
server_key_iv: "308009954eb05b2faad97be47e1b7899"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.088111 14865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "f2d57782ea3f4fc2a043a4faa581572f"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "69e78f544373ab93b40d037de150299d"
server_key_iv: "308009954eb05b2faad97be47e1b7899"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.089530 14865 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:10:33.090456 14880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:33.090660 14865 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:33.090777 14865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "f2d57782ea3f4fc2a043a4faa581572f"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "69e78f544373ab93b40d037de150299d"
server_key_iv: "308009954eb05b2faad97be47e1b7899"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.090860 14865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:33.111143 14865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:33.111547 14865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:33.111725 14865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:33.116398 14865 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:34519
I20260809 02:10:33.116443 14933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:34519 every 8 connection(s)
I20260809 02:10:33.116842 14865 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:10:33.117682 14934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:33.120589 14934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f: Bootstrap starting.
I20260809 02:10:33.121284 14934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:33.121690 14934 log.cc:824] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:33.122470 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 14865
I20260809 02:10:33.122555 14934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f: No bootstrap required, opened a new log
I20260809 02:10:33.122581 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:10:33.122789 5158 external_mini_cluster.cc:1428] Setting key 43cda57e695981b99e272957cb7a03b7
I20260809 02:10:33.124593 14934 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2d57782ea3f4fc2a043a4faa581572f" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34519 } }
I20260809 02:10:33.124785 14934 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:33.124826 14934 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d57782ea3f4fc2a043a4faa581572f, State: Initialized, Role: FOLLOWER
I20260809 02:10:33.125020 14934 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2d57782ea3f4fc2a043a4faa581572f" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34519 } }
I20260809 02:10:33.125119 14934 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:10:33.125145 14934 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:10:33.125219 14934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:33.125908 14934 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2d57782ea3f4fc2a043a4faa581572f" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34519 } }
I20260809 02:10:33.126005 14934 leader_election.cc:302] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2d57782ea3f4fc2a043a4faa581572f; no voters:
I20260809 02:10:33.126231 14934 leader_election.cc:288] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:10:33.126266 14939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:33.126384 14939 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [term 1 LEADER]: Becoming Leader. State: Replica: f2d57782ea3f4fc2a043a4faa581572f, State: Running, Role: LEADER
I20260809 02:10:33.126536 14939 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2d57782ea3f4fc2a043a4faa581572f" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34519 } }
I20260809 02:10:33.126541 14934 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:33.126991 14940 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2d57782ea3f4fc2a043a4faa581572f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2d57782ea3f4fc2a043a4faa581572f" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34519 } } }
I20260809 02:10:33.127211 14940 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:33.127032 14941 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2d57782ea3f4fc2a043a4faa581572f. Latest consensus state: current_term: 1 leader_uuid: "f2d57782ea3f4fc2a043a4faa581572f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2d57782ea3f4fc2a043a4faa581572f" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34519 } } }
I20260809 02:10:33.127467 14941 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:33.127444 14947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:10:33.128126 14947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:10:33.130095 14947 catalog_manager.cc:1347] Generated new cluster ID: a4460542808f407cbd109118903d0514
I20260809 02:10:33.130164 14947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:10:33.130363 14938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:33.123409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33916 (local address 127.5.9.190:34519)
0809 02:10:33.123653 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:33.123661 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:33.123688 (+ 27us) server_negotiation.cc:407] Connection header received
0809 02:10:33.124564 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:33.124571 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:33.124639 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:33.124764 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:33.125062 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:33.126745 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:33.127939 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:33.128539 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:33.128673 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:33.129427 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:33.129431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:33.129431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:33.129679 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:33.129721 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:33.129737 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:33.129808 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:33.129919 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 02:10:33.130012 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":112,"threads_started":1}
I20260809 02:10:33.144055 14947 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:10:33.144851 14947 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:10:33.154583 14947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2d57782ea3f4fc2a043a4faa581572f: Generated new TSK 0
I20260809 02:10:33.154830 14947 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:10:33.156741 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:34519
--builtin_ntp_servers=127.5.9.148:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 02:10:33.258460 14958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:33.258738 14958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:33.258780 14958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:33.260071 14958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:33.260146 14958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:33.260188 14958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:10:33.261735 14958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:34519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:33.262027 14958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:33.262359 14958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:10:33.268075 14958 server_base.cc:1034] running on GCE node
W20260809 02:10:33.268026 14963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:33.268123 14964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:33.268028 14966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:33.268582 14958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:33.268896 14958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:33.270078 14958 hybrid_clock.cc:648] HybridClock initialized: now 1786241433270062 us; error 32 us; skew 500 ppm
I20260809 02:10:33.271500 14958 webserver.cc:460] Webserver started at http://127.5.9.129:37687/ using document root <none> and password file <none>
I20260809 02:10:33.271764 14958 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:33.271837 14958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:33.272003 14958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:33.273182 14958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "422bb53cdde044a494691479b7897ae3"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "c0de65cab19f18370654932f462016c0"
server_key_iv: "55e970c68d4204b6b6d0486d44b67001"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.273656 14958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "422bb53cdde044a494691479b7897ae3"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "c0de65cab19f18370654932f462016c0"
server_key_iv: "55e970c68d4204b6b6d0486d44b67001"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.275611 14958 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:10:33.276549 14972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:33.276815 14958 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:33.276934 14958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "422bb53cdde044a494691479b7897ae3"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "c0de65cab19f18370654932f462016c0"
server_key_iv: "55e970c68d4204b6b6d0486d44b67001"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.277029 14958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:33.288141 14958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:33.288542 14958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:33.288730 14958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:33.289014 14958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:33.289403 14958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:33.289469 14958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:33.289521 14958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:33.289546 14958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:33.296273 14958 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:46231
I20260809 02:10:33.296375 15090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:46231 every 8 connection(s)
I20260809 02:10:33.296742 14958 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:10:33.303026 14938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:33.297980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:47303 (local address 127.5.9.190:34519)
0809 02:10:33.298073 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:33.298075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:33.298356 (+ 281us) server_negotiation.cc:407] Connection header received
0809 02:10:33.298938 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:33.298939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:33.298963 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:33.299029 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:33.299746 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:33.300284 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:33.301359 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:33.301590 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:33.301694 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:33.302669 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:33.302671 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:33.302671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:33.302678 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:33.302696 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:33.302708 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:33.302778 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:33.302891 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 02:10:33.302931 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:10:33.303259 15093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:33.297959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:34519 (local address 127.5.9.129:47303)
0809 02:10:33.298221 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:33.298260 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:33.298821 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:33.299110 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:33.299114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:33.299123 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:33.299643 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:33.299647 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:33.300415 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:33.300416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:33.301252 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:33.301254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:33.301709 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:33.301709 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:33.301853 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:33.302473 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:10:33.302473 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:33.302506 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:10:33.302541 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:33.302781 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:33.302788 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:33.302792 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:10:33.302880 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 02:10:33.303010 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":107,"threads_started":1}
I20260809 02:10:33.304196 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 14958
I20260809 02:10:33.304330 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:10:33.304388 15091 heartbeater.cc:347] Connected to a master server at 127.5.9.190:34519
I20260809 02:10:33.304584 15091 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:33.304636 5158 external_mini_cluster.cc:1428] Setting key eaf44fe09bb5321d2c7eb9056c0a3cea
I20260809 02:10:33.304919 15091 heartbeater.cc:511] Master 127.5.9.190:34519 requested a full tablet report, sending...
I20260809 02:10:33.305838 14896 ts_manager.cc:194] Registered new tserver with Master: 422bb53cdde044a494691479b7897ae3 (127.5.9.129:46231)
I20260809 02:10:33.306413 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:34519
--builtin_ntp_servers=127.5.9.148:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 02:10:33.307071 14896 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.9.129:47303
W20260809 02:10:33.392890 15094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:33.393124 15094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:33.393148 15094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:33.394521 15094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:33.394569 15094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:33.394598 15094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:10:33.396111 15094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:34519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:33.396396 15094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:33.396726 15094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:33.402643 15102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:33.402837 15100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:33.402663 15099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:33.402844 15094 server_base.cc:1034] running on GCE node
I20260809 02:10:33.403175 15094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:33.403470 15094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:33.404673 15094 hybrid_clock.cc:648] HybridClock initialized: now 1786241433404647 us; error 42 us; skew 500 ppm
I20260809 02:10:33.406308 15094 webserver.cc:460] Webserver started at http://127.5.9.130:46275/ using document root <none> and password file <none>
I20260809 02:10:33.406731 15094 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:33.406826 15094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:33.407018 15094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:33.408210 15094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "f04c88a3e4cc4217b3600b1c97f87c58"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "ebb30ea2bd33527458fda9cf12291d6c"
server_key_iv: "8869c269881b1b3a572e272fdeab1de4"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.408612 15094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "f04c88a3e4cc4217b3600b1c97f87c58"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "ebb30ea2bd33527458fda9cf12291d6c"
server_key_iv: "8869c269881b1b3a572e272fdeab1de4"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.410168 15094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260809 02:10:33.411062 15108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:33.411250 15094 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:10:33.411365 15094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "f04c88a3e4cc4217b3600b1c97f87c58"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "ebb30ea2bd33527458fda9cf12291d6c"
server_key_iv: "8869c269881b1b3a572e272fdeab1de4"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.411448 15094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:33.450698 15094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:33.451093 15094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:33.451269 15094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:33.451546 15094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:33.451931 15094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:33.451992 15094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:33.452030 15094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:33.452078 15094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:33.458384 15094 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:36965
I20260809 02:10:33.458447 15227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:36965 every 8 connection(s)
I20260809 02:10:33.458801 15094 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:10:33.463792 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 15094
I20260809 02:10:33.463912 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:10:33.464167 5158 external_mini_cluster.cc:1428] Setting key c19924889719785e72d783e538033746
I20260809 02:10:33.464541 14938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:33.459562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:47373 (local address 127.5.9.190:34519)
0809 02:10:33.459664 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:33.459667 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:33.459880 (+ 213us) server_negotiation.cc:407] Connection header received
0809 02:10:33.460458 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:33.460459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:33.460470 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:33.460538 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:33.461259 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:33.461809 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:33.462890 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:33.463091 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:33.463146 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:33.464105 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:33.464106 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:33.464107 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:33.464112 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:33.464133 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:33.464138 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:33.464203 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:33.464360 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 02:10:33.464426 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:10:33.464747 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:33.459520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:34519 (local address 127.5.9.130:47373)
0809 02:10:33.459743 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:33.459781 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:33.460326 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:33.460588 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:33.460592 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:33.460602 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:33.461145 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:33.461150 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:33.461938 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:33.461940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:33.462781 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:33.462784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:33.463172 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:33.463173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:33.463293 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:33.463914 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:10:33.463915 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:33.463957 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:10:33.463993 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:33.464221 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:33.464228 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:33.464232 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:10:33.464352 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 02:10:33.464486 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":113,"threads_started":1}
I20260809 02:10:33.465231 15228 heartbeater.cc:347] Connected to a master server at 127.5.9.190:34519
I20260809 02:10:33.465324 15228 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:33.465504 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:34519
--builtin_ntp_servers=127.5.9.148:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 02:10:33.465559 15228 heartbeater.cc:511] Master 127.5.9.190:34519 requested a full tablet report, sending...
I20260809 02:10:33.466041 14896 ts_manager.cc:194] Registered new tserver with Master: f04c88a3e4cc4217b3600b1c97f87c58 (127.5.9.130:36965)
I20260809 02:10:33.466562 14896 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.9.130:47373
W20260809 02:10:33.552021 15231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:33.552277 15231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:33.552302 15231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:33.553525 15231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:33.553570 15231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:33.553596 15231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:10:33.555127 15231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:34519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:33.555356 15231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:33.555634 15231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:33.561065 15240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:33.561125 15237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:33.561118 15238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:33.561424 15231 server_base.cc:1034] running on GCE node
I20260809 02:10:33.561733 15231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:33.561975 15231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:33.563158 15231 hybrid_clock.cc:648] HybridClock initialized: now 1786241433563136 us; error 45 us; skew 500 ppm
I20260809 02:10:33.564735 15231 webserver.cc:460] Webserver started at http://127.5.9.131:36179/ using document root <none> and password file <none>
I20260809 02:10:33.565017 15231 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:33.565069 15231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:33.565258 15231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:33.566488 15231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "06c06d378ebe40de9da73693019254bb"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "339b9c2fe282fd5001c29fe55c5873f3"
server_key_iv: "98029b36c4296bdc919537807d57a890"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.566902 15231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "06c06d378ebe40de9da73693019254bb"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "339b9c2fe282fd5001c29fe55c5873f3"
server_key_iv: "98029b36c4296bdc919537807d57a890"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.568459 15231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:10:33.569397 15246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:33.569604 15231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:10:33.569742 15231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "06c06d378ebe40de9da73693019254bb"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "339b9c2fe282fd5001c29fe55c5873f3"
server_key_iv: "98029b36c4296bdc919537807d57a890"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.569805 15231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:33.597862 15231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:33.598165 15231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:33.598275 15231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:33.598625 15231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:33.598977 15231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:33.599013 15231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:33.599037 15231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:33.599050 15231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:33.605196 15231 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:32795
I20260809 02:10:33.605242 15363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:32795 every 8 connection(s)
I20260809 02:10:33.605607 15231 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:10:33.611318 14938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:33.606479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:33475 (local address 127.5.9.190:34519)
0809 02:10:33.606571 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:33.606574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:33.606808 (+ 234us) server_negotiation.cc:407] Connection header received
0809 02:10:33.607336 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:33.607337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:33.607349 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:33.607400 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:33.608101 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:33.608645 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:33.609742 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:33.609974 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:33.610040 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:33.611016 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:33.611017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:33.611018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:33.611022 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:33.611039 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:33.611043 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:33.611098 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:33.611207 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 02:10:33.611240 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 02:10:33.611467 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:33.606436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:34519 (local address 127.5.9.131:33475)
0809 02:10:33.606674 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:33.606712 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:33.607236 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:33.607468 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:33.607471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:33.607479 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:33.608015 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:33.608020 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:33.608738 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:33.608738 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:33.609584 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:33.609586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:33.610061 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:33.610062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:33.610182 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:33.610852 (+ 670us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:10:33.610853 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:33.610885 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:10:33.610910 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:33.611105 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:33.611107 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:33.611110 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:10:33.611161 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 02:10:33.611249 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":123,"threads_started":1}
I20260809 02:10:33.612058 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 15231
I20260809 02:10:33.612192 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:10:33.612185 15364 heartbeater.cc:347] Connected to a master server at 127.5.9.190:34519
I20260809 02:10:33.612308 15364 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:33.612418 5158 external_mini_cluster.cc:1428] Setting key 19b1b605c8a8d77a2be8b5cf767259d9
I20260809 02:10:33.612568 15364 heartbeater.cc:511] Master 127.5.9.190:34519 requested a full tablet report, sending...
I20260809 02:10:33.613003 14896 ts_manager.cc:194] Registered new tserver with Master: 06c06d378ebe40de9da73693019254bb (127.5.9.131:32795)
I20260809 02:10:33.613425 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:10:33.613445 14896 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.9.131:33475
W20260809 02:10:33.615746 14938 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:33.615195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33928 (local address 127.5.9.190:34519)
0809 02:10:33.615282 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:33.615284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:33.615384 (+ 100us) server_negotiation.cc:407] Connection header received
0809 02:10:33.615449 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:33.615450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:33.615463 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:33.615521 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:33.615668 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33928: BlockingRecv error: recv got EOF from 127.0.0.1:33928 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 02:10:33.615784 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 14958
I20260809 02:10:33.624969 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 15094
I20260809 02:10:33.632282 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 15231
I20260809 02:10:33.638819 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 14865
2026-08-09T02:10:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (682 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T02:10:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:10:33Z Disabled control of system clock
I20260809 02:10:33.669764 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:44435
--webserver_interface=127.5.9.190
--webserver_port=0
--builtin_ntp_servers=127.5.9.148:34501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:44435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 02:10:33.757891 15376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:33.758147 15376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:33.758172 15376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:33.759322 15376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:33.759364 15376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:33.759377 15376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:33.759390 15376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:33.759402 15376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:10:33.761016 15376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:44435
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:44435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:33.761368 15376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:33.761744 15376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:33.767038 15381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:33.767105 15384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:33.767038 15382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:33.767367 15376 server_base.cc:1034] running on GCE node
I20260809 02:10:33.767699 15376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:33.768011 15376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:33.769204 15376 hybrid_clock.cc:648] HybridClock initialized: now 1786241433769189 us; error 43 us; skew 500 ppm
I20260809 02:10:33.770839 15376 webserver.cc:460] Webserver started at http://127.5.9.190:35487/ using document root <none> and password file <none>
I20260809 02:10:33.771147 15376 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:33.771199 15376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:33.771299 15376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:33.772387 15376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "7678374046e943dd9513ba6ab5eacfb7"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "58db75fbbcfcbdb81266dc8c40747b20"
server_key_iv: "68869230510120bb035c57295721f8f3"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.772716 15376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "7678374046e943dd9513ba6ab5eacfb7"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "58db75fbbcfcbdb81266dc8c40747b20"
server_key_iv: "68869230510120bb035c57295721f8f3"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.774296 15376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:10:33.775132 15390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:33.775408 15376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:10:33.775525 15376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "7678374046e943dd9513ba6ab5eacfb7"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "58db75fbbcfcbdb81266dc8c40747b20"
server_key_iv: "68869230510120bb035c57295721f8f3"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.775619 15376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:33.779335 15376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:33.779640 15376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:33.779793 15376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:33.784291 15376 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:44435
I20260809 02:10:33.784327 15447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:44435 every 8 connection(s)
I20260809 02:10:33.784734 15376 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:10:33.785797 15448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:33.788359 15448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7: Bootstrap starting.
I20260809 02:10:33.789038 15448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:33.789358 15448 log.cc:824] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:33.790266 15448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7: No bootstrap required, opened a new log
I20260809 02:10:33.791975 15448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7678374046e943dd9513ba6ab5eacfb7" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44435 } }
I20260809 02:10:33.792129 15448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:33.792153 15448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7678374046e943dd9513ba6ab5eacfb7, State: Initialized, Role: FOLLOWER
I20260809 02:10:33.792313 15448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7678374046e943dd9513ba6ab5eacfb7" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44435 } }
I20260809 02:10:33.792394 15448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:10:33.792418 15448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:10:33.792470 15448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:33.793071 15448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7678374046e943dd9513ba6ab5eacfb7" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44435 } }
I20260809 02:10:33.793166 15448 leader_election.cc:302] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7678374046e943dd9513ba6ab5eacfb7; no voters:
I20260809 02:10:33.793359 15448 leader_election.cc:288] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:10:33.793416 15451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:33.793666 15451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [term 1 LEADER]: Becoming Leader. State: Replica: 7678374046e943dd9513ba6ab5eacfb7, State: Running, Role: LEADER
I20260809 02:10:33.793736 15448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:33.793769 15451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7678374046e943dd9513ba6ab5eacfb7" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44435 } }
I20260809 02:10:33.794226 15453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7678374046e943dd9513ba6ab5eacfb7. Latest consensus state: current_term: 1 leader_uuid: "7678374046e943dd9513ba6ab5eacfb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7678374046e943dd9513ba6ab5eacfb7" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44435 } } }
I20260809 02:10:33.794422 15453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:33.794319 15452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7678374046e943dd9513ba6ab5eacfb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7678374046e943dd9513ba6ab5eacfb7" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44435 } } }
I20260809 02:10:33.794710 15452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7 [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:33.794937 15461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:10:33.794999 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 15376
I20260809 02:10:33.795113 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:10:33.795344 5158 external_mini_cluster.cc:1428] Setting key 72f15fd196d69792384cf6a66a5e510a
I20260809 02:10:33.795508 15461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:10:33.797559 15461 catalog_manager.cc:1347] Generated new cluster ID: d305321c9bde458e9871e2a0e2d2d948
I20260809 02:10:33.797645 15461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:10:33.798245 15471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:33.796129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46354 (local address 127.5.9.190:44435)
0809 02:10:33.796557 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:33.796568 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:33.796602 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:10:33.796724 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:33.796731 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:33.796777 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:33.797108 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:33.797753 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:33.797757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:33.797757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:33.797761 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:33.797813 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:33.797829 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:33.797895 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:33.798013 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 02:10:33.798072 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":81,"threads_started":1}
I20260809 02:10:33.831339 15461 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:10:33.831560 15461 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:10:33.841274 15461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7678374046e943dd9513ba6ab5eacfb7: Generated new TSK 0
I20260809 02:10:33.841547 15461 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:10:33.850067 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:44435
--builtin_ntp_servers=127.5.9.148:34501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 02:10:33.938082 15473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:33.938315 15473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:33.938339 15473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:33.939610 15473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:33.939657 15473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:33.939684 15473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:10:33.941119 15473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:44435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:33.941395 15473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:33.941782 15473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:33.947254 15482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:33.947249 15480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:33.947249 15479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:33.947582 15473 server_base.cc:1034] running on GCE node
I20260809 02:10:33.947855 15473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:33.948125 15473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:33.949299 15473 hybrid_clock.cc:648] HybridClock initialized: now 1786241433949293 us; error 38 us; skew 500 ppm
I20260809 02:10:33.950773 15473 webserver.cc:460] Webserver started at http://127.5.9.129:40395/ using document root <none> and password file <none>
I20260809 02:10:33.951160 15473 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:33.951237 15473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:33.951388 15473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:33.952554 15473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "50e2e62ba0bf428d9823dc37a3157dcb"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "c1177948a70efe13cff935c6ddbb0e88"
server_key_iv: "b43da5198a42ccf922a429083f9b0583"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.952960 15473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "50e2e62ba0bf428d9823dc37a3157dcb"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "c1177948a70efe13cff935c6ddbb0e88"
server_key_iv: "b43da5198a42ccf922a429083f9b0583"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.954484 15473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:10:33.955273 15488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:33.955482 15473 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:10:33.955595 15473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "50e2e62ba0bf428d9823dc37a3157dcb"
format_stamp: "Formatted at 2026-08-09 02:10:33 on dist-test-slave-p3t5"
server_key: "c1177948a70efe13cff935c6ddbb0e88"
server_key_iv: "b43da5198a42ccf922a429083f9b0583"
server_key_version: "encryptionkey@0"
I20260809 02:10:33.955678 15473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:33.959395 15473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:33.959702 15473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:33.959859 15473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:33.960109 15473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:33.960451 15473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:33.960508 15473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:33.960552 15473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:33.960580 15473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:33.967203 15473 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:43161
I20260809 02:10:33.967254 15604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:43161 every 8 connection(s)
I20260809 02:10:33.967630 15473 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:10:33.970960 15471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:33.968424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:59079 (local address 127.5.9.190:44435)
0809 02:10:33.968485 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:33.968487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:33.968750 (+ 263us) server_negotiation.cc:407] Connection header received
0809 02:10:33.969307 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:33.969308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:33.969316 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:33.969376 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:33.970652 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:33.970654 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:33.970655 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:33.970659 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:33.970680 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:33.970684 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:33.970765 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:33.970864 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 02:10:33.970881 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 02:10:33.971179 15607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:33.968412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:44435 (local address 127.5.9.129:59079)
0809 02:10:33.968628 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:33.968664 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:33.969193 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:33.969468 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:33.969472 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:33.969481 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:33.970492 (+ 1011us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:10:33.970493 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:33.970524 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:10:33.970554 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:33.970761 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:33.970769 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:33.970773 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:10:33.970877 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 02:10:33.970943 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":101,"threads_started":1}
I20260809 02:10:33.971700 15605 heartbeater.cc:347] Connected to a master server at 127.5.9.190:44435
I20260809 02:10:33.971784 15605 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:33.971958 15605 heartbeater.cc:511] Master 127.5.9.190:44435 requested a full tablet report, sending...
I20260809 02:10:33.972517 15408 ts_manager.cc:194] Registered new tserver with Master: 50e2e62ba0bf428d9823dc37a3157dcb (127.5.9.129:43161)
I20260809 02:10:33.976693 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 15473
I20260809 02:10:33.976817 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:10:33.977033 5158 external_mini_cluster.cc:1428] Setting key eb3d53628d24d439e5d31fecf79124a2
I20260809 02:10:33.978058 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:44435
--builtin_ntp_servers=127.5.9.148:34501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 02:10:34.067243 15608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:34.067473 15608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:34.067499 15608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:34.068739 15608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:34.068789 15608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:34.068817 15608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:10:34.070379 15608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:44435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:34.070659 15608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:34.070990 15608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:34.076279 15614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:34.076318 15615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:34.076287 15617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:34.076318 15608 server_base.cc:1034] running on GCE node
I20260809 02:10:34.076668 15608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:34.076882 15608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:34.078024 15608 hybrid_clock.cc:648] HybridClock initialized: now 1786241434078008 us; error 31 us; skew 500 ppm
I20260809 02:10:34.079526 15608 webserver.cc:460] Webserver started at http://127.5.9.130:45415/ using document root <none> and password file <none>
I20260809 02:10:34.079800 15608 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:34.079870 15608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:34.080037 15608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:34.081154 15608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "0ddc97294ca54364bd52eb985c450f9d"
format_stamp: "Formatted at 2026-08-09 02:10:34 on dist-test-slave-p3t5"
server_key: "615941bc763584e75395cf5838c4a732"
server_key_iv: "cb04d62df4f451311279242773a00a49"
server_key_version: "encryptionkey@0"
I20260809 02:10:34.081537 15608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "0ddc97294ca54364bd52eb985c450f9d"
format_stamp: "Formatted at 2026-08-09 02:10:34 on dist-test-slave-p3t5"
server_key: "615941bc763584e75395cf5838c4a732"
server_key_iv: "cb04d62df4f451311279242773a00a49"
server_key_version: "encryptionkey@0"
I20260809 02:10:34.083027 15608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:10:34.083930 15623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:34.084156 15608 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:10:34.084264 15608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "0ddc97294ca54364bd52eb985c450f9d"
format_stamp: "Formatted at 2026-08-09 02:10:34 on dist-test-slave-p3t5"
server_key: "615941bc763584e75395cf5838c4a732"
server_key_iv: "cb04d62df4f451311279242773a00a49"
server_key_version: "encryptionkey@0"
I20260809 02:10:34.084347 15608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:34.088001 15608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:34.088297 15608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:34.088447 15608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:34.088660 15608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:34.088994 15608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:34.089027 15608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:34.089085 15608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:34.089113 15608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:34.095585 15608 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:39527
I20260809 02:10:34.095654 15739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:39527 every 8 connection(s)
I20260809 02:10:34.095993 15608 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:10:34.099273 15471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:34.096888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:37929 (local address 127.5.9.190:44435)
0809 02:10:34.096939 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:34.096942 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:34.097222 (+ 280us) server_negotiation.cc:407] Connection header received
0809 02:10:34.097767 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:34.097769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:34.097774 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:34.097815 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:34.099062 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:34.099063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:34.099063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:34.099067 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:34.099085 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:34.099089 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:34.099142 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:34.099199 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 02:10:34.099213 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:10:34.099475 15742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:34.096896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:44435 (local address 127.5.9.130:37929)
0809 02:10:34.097096 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:34.097132 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:34.097609 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:34.097884 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:34.097888 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:34.097897 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:34.098913 (+ 1016us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:10:34.098913 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:34.098945 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:10:34.098975 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:34.099138 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:34.099140 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:34.099143 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:10:34.099196 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 02:10:34.099252 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
I20260809 02:10:34.100111 15740 heartbeater.cc:347] Connected to a master server at 127.5.9.190:44435
I20260809 02:10:34.100227 15740 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:34.100387 15740 heartbeater.cc:511] Master 127.5.9.190:44435 requested a full tablet report, sending...
I20260809 02:10:34.100760 15408 ts_manager.cc:194] Registered new tserver with Master: 0ddc97294ca54364bd52eb985c450f9d (127.5.9.130:39527)
I20260809 02:10:34.104214 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 15608
I20260809 02:10:34.104312 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:10:34.104466 5158 external_mini_cluster.cc:1428] Setting key 4b736b965c1faecd79bfe57212ee8d18
I20260809 02:10:34.105397 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:44435
--builtin_ntp_servers=127.5.9.148:34501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 02:10:34.192013 15743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:34.192234 15743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:34.192257 15743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:34.193560 15743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:34.193650 15743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:34.193696 15743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:10:34.195374 15743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:44435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:34.195642 15743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:34.195971 15743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:34.201207 15751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:34.201237 15749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:34.201258 15748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:34.201555 15743 server_base.cc:1034] running on GCE node
I20260809 02:10:34.201843 15743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:34.202102 15743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:34.203291 15743 hybrid_clock.cc:648] HybridClock initialized: now 1786241434203284 us; error 37 us; skew 500 ppm
I20260809 02:10:34.204753 15743 webserver.cc:460] Webserver started at http://127.5.9.131:37815/ using document root <none> and password file <none>
I20260809 02:10:34.205071 15743 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:34.205150 15743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:34.205315 15743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:34.206526 15743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "37441a5be0824e76af0b3f8fe8c98853"
format_stamp: "Formatted at 2026-08-09 02:10:34 on dist-test-slave-p3t5"
server_key: "5fcb0bbde9db949499580ac0fe1416ad"
server_key_iv: "8a6269f37eb1d453fa0c85f31dc0b594"
server_key_version: "encryptionkey@0"
I20260809 02:10:34.206931 15743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "37441a5be0824e76af0b3f8fe8c98853"
format_stamp: "Formatted at 2026-08-09 02:10:34 on dist-test-slave-p3t5"
server_key: "5fcb0bbde9db949499580ac0fe1416ad"
server_key_iv: "8a6269f37eb1d453fa0c85f31dc0b594"
server_key_version: "encryptionkey@0"
I20260809 02:10:34.208346 15743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:10:34.209175 15758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:34.209355 15743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:34.209458 15743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "37441a5be0824e76af0b3f8fe8c98853"
format_stamp: "Formatted at 2026-08-09 02:10:34 on dist-test-slave-p3t5"
server_key: "5fcb0bbde9db949499580ac0fe1416ad"
server_key_iv: "8a6269f37eb1d453fa0c85f31dc0b594"
server_key_version: "encryptionkey@0"
I20260809 02:10:34.209547 15743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:34.213093 15743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:34.213384 15743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:34.213537 15743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:34.213770 15743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:34.214104 15743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:34.214161 15743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:34.214202 15743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:34.214228 15743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:34.220081 15743 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:42907
I20260809 02:10:34.220153 15872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:42907 every 8 connection(s)
I20260809 02:10:34.220510 15743 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:10:34.221334 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 15743
I20260809 02:10:34.221452 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:10:34.221696 5158 external_mini_cluster.cc:1428] Setting key 75e12197c3f1bebeb37220ead43e3c87
I20260809 02:10:34.223832 15471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:34.221339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:55015 (local address 127.5.9.190:44435)
0809 02:10:34.221416 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:34.221419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:34.221637 (+ 218us) server_negotiation.cc:407] Connection header received
0809 02:10:34.222301 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:34.222302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:34.222310 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:34.222358 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:34.223602 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:34.223604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:34.223604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:10:34.223608 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:34.223624 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:10:34.223629 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:34.223678 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:34.223761 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 02:10:34.223776 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:10:34.224007 15875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:34.221283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:44435 (local address 127.5.9.131:55015)
0809 02:10:34.221475 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:34.221518 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:34.222185 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:34.222425 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:34.222430 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:34.222440 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:34.223433 (+ 993us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:10:34.223433 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:34.223462 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:10:34.223490 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:34.223692 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:34.223697 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:34.223700 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:10:34.223759 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 02:10:34.223811 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260809 02:10:34.224519 15873 heartbeater.cc:347] Connected to a master server at 127.5.9.190:44435
I20260809 02:10:34.224615 15873 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:34.224783 15873 heartbeater.cc:511] Master 127.5.9.190:44435 requested a full tablet report, sending...
I20260809 02:10:34.225154 15408 ts_manager.cc:194] Registered new tserver with Master: 37441a5be0824e76af0b3f8fe8c98853 (127.5.9.131:42907)
I20260809 02:10:34.233978 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 02:10:34.236143 15471 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:10:34.235529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46364 (local address 127.5.9.190:44435)
0809 02:10:34.235614 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:34.235617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:34.235713 (+ 96us) server_negotiation.cc:407] Connection header received
0809 02:10:34.235813 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:34.235814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:34.235820 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:34.235891 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:34.236046 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46364: BlockingRecv error: recv got EOF from 127.0.0.1:46364 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:10:34.236240 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 15473
I20260809 02:10:34.243096 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 15608
I20260809 02:10:34.249316 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 15743
I20260809 02:10:34.255779 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 15376
2026-08-09T02:10:34Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (616 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:10:34 dist-test-slave-p3t5 krb5kdc[15883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:10:34 dist-test-slave-p3t5 krb5kdc[15883](info): set up 2 sockets
Aug 09 02:10:34 dist-test-slave-p3t5 krb5kdc[15883](info): commencing operation
krb5kdc: starting...
W20260809 02:10:35.343659 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:10:35 dist-test-slave-p3t5 krb5kdc[15883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:10:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:10:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:10:35.485111 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:33497
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:45057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:33497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:35.570603 15897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:35.570915 15897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:35.570974 15897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:35.572292 15897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:10:35.572378 15897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:35.572405 15897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:35.572439 15897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:10:35.572463 15897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:10:35.574225 15897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:45057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:33497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:33497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:35.574542 15897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:35.574801 15897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:35.580329 15904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:35.580324 15903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:35.580324 15906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:35.580371 15897 server_base.cc:1034] running on GCE node
I20260809 02:10:35.580977 15897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:35.581307 15897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:35.582468 15897 hybrid_clock.cc:648] HybridClock initialized: now 1786241435582452 us; error 33 us; skew 500 ppm
Aug 09 02:10:35 dist-test-slave-p3t5 krb5kdc[15883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241435, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:35.585124 15897 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:10:35.585726 15897 webserver.cc:460] Webserver started at http://127.5.9.190:40485/ using document root <none> and password file <none>
I20260809 02:10:35.585983 15897 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:35.586053 15897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:35.586206 15897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:35.587342 15897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "2e2ae3ba763443f3abd170c949e7f50e"
format_stamp: "Formatted at 2026-08-09 02:10:35 on dist-test-slave-p3t5"
server_key: "5b3954297b7bda73c528f2bd742026b1"
server_key_iv: "f9640ba7ae4f70866572680600526814"
server_key_version: "encryptionkey@0"
I20260809 02:10:35.587739 15897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "2e2ae3ba763443f3abd170c949e7f50e"
format_stamp: "Formatted at 2026-08-09 02:10:35 on dist-test-slave-p3t5"
server_key: "5b3954297b7bda73c528f2bd742026b1"
server_key_iv: "f9640ba7ae4f70866572680600526814"
server_key_version: "encryptionkey@0"
I20260809 02:10:35.589190 15897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:10:35.590106 15913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:35.590335 15897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:10:35.590447 15897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "2e2ae3ba763443f3abd170c949e7f50e"
format_stamp: "Formatted at 2026-08-09 02:10:35 on dist-test-slave-p3t5"
server_key: "5b3954297b7bda73c528f2bd742026b1"
server_key_iv: "f9640ba7ae4f70866572680600526814"
server_key_version: "encryptionkey@0"
I20260809 02:10:35.590528 15897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:35.600960 15897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:35.603627 15897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:35.603816 15897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:35.608570 15897 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:33497
I20260809 02:10:35.608623 15966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:33497 every 8 connection(s)
I20260809 02:10:35.609030 15897 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:10:35.610162 15967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:35.611532 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 15897
I20260809 02:10:35.611624 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:10:35.611840 5158 external_mini_cluster.cc:1428] Setting key 71137e035151f059ef02d8975e0a0c9b
I20260809 02:10:35.613220 15967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e: Bootstrap starting.
I20260809 02:10:35.614180 15967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:35.614532 15967 log.cc:824] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:35.615396 15967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e: No bootstrap required, opened a new log
I20260809 02:10:35.618182 15967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2ae3ba763443f3abd170c949e7f50e" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33497 } }
I20260809 02:10:35.618417 15967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:35.618460 15967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e2ae3ba763443f3abd170c949e7f50e, State: Initialized, Role: FOLLOWER
I20260809 02:10:35.618641 15967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2ae3ba763443f3abd170c949e7f50e" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33497 } }
I20260809 02:10:35.618772 15967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:10:35.618820 15967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:10:35.618873 15967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:35.619711 15967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2ae3ba763443f3abd170c949e7f50e" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33497 } }
Aug 09 02:10:35 dist-test-slave-p3t5 krb5kdc[15883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:10:35.619818 15967 leader_election.cc:302] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e2ae3ba763443f3abd170c949e7f50e; no voters:
I20260809 02:10:35.620075 15967 leader_election.cc:288] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:10:35.620105 15972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:35.620247 15972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [term 1 LEADER]: Becoming Leader. State: Replica: 2e2ae3ba763443f3abd170c949e7f50e, State: Running, Role: LEADER
I20260809 02:10:35.620438 15967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:10:35.620419 15972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2ae3ba763443f3abd170c949e7f50e" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33497 } }
I20260809 02:10:35.620797 15972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e2ae3ba763443f3abd170c949e7f50e. Latest consensus state: current_term: 1 leader_uuid: "2e2ae3ba763443f3abd170c949e7f50e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2ae3ba763443f3abd170c949e7f50e" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33497 } } }
I20260809 02:10:35.620893 15972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:35.621166 15975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e2ae3ba763443f3abd170c949e7f50e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2ae3ba763443f3abd170c949e7f50e" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33497 } } }
I20260809 02:10:35.621325 15975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e [sys.catalog]: This master's current role is: LEADER
I20260809 02:10:35.621635 15978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:10:35.623208 15978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:10:35.624840 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:35.612487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41622 (local address 127.5.9.190:33497)
0809 02:10:35.612675 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:35.612683 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:35.612711 (+ 28us) server_negotiation.cc:407] Connection header received
0809 02:10:35.613130 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:35.613136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:35.613185 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:35.613311 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:35.614351 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:35.615610 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:35.616786 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:35.617218 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:35.617317 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:35.620492 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:35.620497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:35.620498 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:35.620512 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:35.622733 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:35.623229 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:35.623235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:35.623236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:35.623404 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:35.623669 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:35.623670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:35.623671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:35.624009 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:35.624189 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:35.624471 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 02:10:35.624634 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260809 02:10:35.625273 15978 catalog_manager.cc:1347] Generated new cluster ID: 38f9f7ff7d654f199aa95b89b2fdca64
I20260809 02:10:35.625360 15978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:10:35.645579 15978 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:10:35.646426 15978 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:10:35.653129 15978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e: Generated new TSK 0
I20260809 02:10:35.653362 15978 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:10:35.685808 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:33497
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:45057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:35.776502 15994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:35.776822 15994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:35.776880 15994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:35.778235 15994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:35.778321 15994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:35.778366 15994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:10:35.780014 15994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:45057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:33497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:35.780334 15994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:35.780611 15994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:35.786131 15999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:35.786132 16000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:35.786158 16002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:35.786475 15994 server_base.cc:1034] running on GCE node
I20260809 02:10:35.786693 15994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:35.786945 15994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:35.788094 15994 hybrid_clock.cc:648] HybridClock initialized: now 1786241435788081 us; error 31 us; skew 500 ppm
Aug 09 02:10:35 dist-test-slave-p3t5 krb5kdc[15883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241435, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:35.790961 15994 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:10:35.791450 15994 webserver.cc:460] Webserver started at http://127.5.9.129:36909/ using document root <none> and password file <none>
I20260809 02:10:35.791723 15994 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:35.791800 15994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:35.791965 15994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:35.793156 15994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "a2ae2c39a294405da6be1d243b474ebf"
format_stamp: "Formatted at 2026-08-09 02:10:35 on dist-test-slave-p3t5"
server_key: "fdb84a09d5b70a23d6a61a8225c8282f"
server_key_iv: "aec401f1184bf511c3f1e8fafbb8276f"
server_key_version: "encryptionkey@0"
I20260809 02:10:35.793622 15994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "a2ae2c39a294405da6be1d243b474ebf"
format_stamp: "Formatted at 2026-08-09 02:10:35 on dist-test-slave-p3t5"
server_key: "fdb84a09d5b70a23d6a61a8225c8282f"
server_key_iv: "aec401f1184bf511c3f1e8fafbb8276f"
server_key_version: "encryptionkey@0"
I20260809 02:10:35.795202 15994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:35.796113 16009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:35.796384 15994 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:35.796512 15994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "a2ae2c39a294405da6be1d243b474ebf"
format_stamp: "Formatted at 2026-08-09 02:10:35 on dist-test-slave-p3t5"
server_key: "fdb84a09d5b70a23d6a61a8225c8282f"
server_key_iv: "aec401f1184bf511c3f1e8fafbb8276f"
server_key_version: "encryptionkey@0"
I20260809 02:10:35.796598 15994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:35.819521 15994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:35.822508 15994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:35.822703 15994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:35.822994 15994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:35.823348 15994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:35.823407 15994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:35.823447 15994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:35.823478 15994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:35.829818 15994 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:42871
I20260809 02:10:35.829917 16124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:42871 every 8 connection(s)
I20260809 02:10:35.830281 15994 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:10:35.832906 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 15994
I20260809 02:10:35.833012 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:10:35.833254 5158 external_mini_cluster.cc:1428] Setting key d7926023ff9d2009fc8c30a80fe20205
Aug 09 02:10:35 dist-test-slave-p3t5 krb5kdc[15883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241435, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:10:35.841455 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:35.831146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:53677 (local address 127.5.9.190:33497)
0809 02:10:35.831239 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:35.831242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:35.831430 (+ 188us) server_negotiation.cc:407] Connection header received
0809 02:10:35.831958 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:35.831960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:35.831971 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:35.832042 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:35.833174 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:35.833963 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:35.835035 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:35.835437 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:35.835525 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:35.838132 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:35.838133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:35.838134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:35.838142 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:35.839878 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:35.840388 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:35.840390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:35.840390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:35.840455 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:35.840731 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:35.840732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:35.840733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:35.840869 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:35.840945 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:35.841252 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 02:10:35.841353 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 02:10:35.841709 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:35.831095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:33497 (local address 127.5.9.129:53677)
0809 02:10:35.831289 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:35.831329 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:35.831835 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:35.832101 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:35.832104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:35.832737 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:35.832956 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:35.832989 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:35.834090 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:35.834091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:35.834931 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:35.834933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:35.835537 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:35.835538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:35.835645 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:35.836276 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:35.836285 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:35.838012 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:35.840006 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:35.840016 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:35.840018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:35.840278 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:35.840528 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:35.840529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:35.840529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:35.840652 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:35.840949 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:35.840957 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:35.841115 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 02:10:35.841228 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 02:10:35.841414 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":110,"threads_started":1}
I20260809 02:10:35.842255 16125 heartbeater.cc:347] Connected to a master server at 127.5.9.190:33497
I20260809 02:10:35.842360 16125 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:35.842577 16125 heartbeater.cc:511] Master 127.5.9.190:33497 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
I20260809 02:10:35.843214 15930 ts_manager.cc:194] Registered new tserver with Master: a2ae2c39a294405da6be1d243b474ebf (127.5.9.129:42871)
I20260809 02:10:35.843863 15930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:53677
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:10:35.864140 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:33497
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:45057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:35.949127 16130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:35.949359 16130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:35.949384 16130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:35.950964 16130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:35.951032 16130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:35.951061 16130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:10:35.952756 16130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:45057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:33497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:35.953071 16130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:35.953331 16130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:10:35.958927 16130 server_base.cc:1034] running on GCE node
W20260809 02:10:35.958770 16136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:35.958762 16135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:35.959013 16138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:35.959371 16130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:35.959694 16130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:35.960865 16130 hybrid_clock.cc:648] HybridClock initialized: now 1786241435960838 us; error 47 us; skew 500 ppm
Aug 09 02:10:35 dist-test-slave-p3t5 krb5kdc[15883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241435, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:35.963755 16130 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:10:35.964254 16130 webserver.cc:460] Webserver started at http://127.5.9.130:38493/ using document root <none> and password file <none>
I20260809 02:10:35.964524 16130 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:35.964596 16130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:35.964757 16130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:35.965968 16130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "af84e1b7696d49aa9b0b3090c86aa7c9"
format_stamp: "Formatted at 2026-08-09 02:10:35 on dist-test-slave-p3t5"
server_key: "b8d2d7269e2a63451f5714858bfd6048"
server_key_iv: "4c22d668141f6318b4ad1de3c12ec526"
server_key_version: "encryptionkey@0"
I20260809 02:10:35.966377 16130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "af84e1b7696d49aa9b0b3090c86aa7c9"
format_stamp: "Formatted at 2026-08-09 02:10:35 on dist-test-slave-p3t5"
server_key: "b8d2d7269e2a63451f5714858bfd6048"
server_key_iv: "4c22d668141f6318b4ad1de3c12ec526"
server_key_version: "encryptionkey@0"
I20260809 02:10:35.967907 16130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:10:35.968815 16145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:35.969007 16130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:10:35.969121 16130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "af84e1b7696d49aa9b0b3090c86aa7c9"
format_stamp: "Formatted at 2026-08-09 02:10:35 on dist-test-slave-p3t5"
server_key: "b8d2d7269e2a63451f5714858bfd6048"
server_key_iv: "4c22d668141f6318b4ad1de3c12ec526"
server_key_version: "encryptionkey@0"
I20260809 02:10:35.969208 16130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:35.982226 16130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:35.984764 16130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:35.984951 16130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:35.985217 16130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:35.985554 16130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:35.985610 16130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:35.985683 16130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:35.985731 16130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:35.992122 16130 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:39835
I20260809 02:10:35.992158 16263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:39835 every 8 connection(s)
I20260809 02:10:35.992517 16130 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:10:35 dist-test-slave-p3t5 krb5kdc[15883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241435, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:10:36.000687 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 16130
I20260809 02:10:36.000865 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:10:36.001091 5158 external_mini_cluster.cc:1428] Setting key 92f8fd0cb400496f357d3eafa1d74a62
I20260809 02:10:36.003185 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:35.993300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:43091 (local address 127.5.9.190:33497)
0809 02:10:35.993351 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:35.993353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:35.993636 (+ 283us) server_negotiation.cc:407] Connection header received
0809 02:10:35.994281 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:35.994282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:35.994294 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:35.994364 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:35.995349 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:35.995867 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:35.996861 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:35.997059 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:35.997120 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:35.999508 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:35.999509 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:35.999509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:35.999517 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:36.001595 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:36.002117 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:36.002119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:36.002119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:36.002184 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:36.002486 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:36.002487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:36.002488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:36.002618 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:36.002692 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:36.002998 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 02:10:36.003090 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:10:36.003422 16266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:35.993297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:33497 (local address 127.5.9.130:43091)
0809 02:10:35.993461 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:35.993497 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:35.994154 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:35.994455 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:35.994459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:35.995044 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:35.995244 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:35.995249 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:35.995955 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:35.995955 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:35.996758 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:35.996760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:35.997141 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:35.997142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:35.997243 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:35.997872 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:35.997880 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:35.999390 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:36.001756 (+ 2366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:36.001773 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:36.001775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:36.002016 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:36.002270 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:36.002271 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:36.002271 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:36.002393 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:36.002697 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:36.002705 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:36.002857 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 02:10:36.002966 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 02:10:36.003144 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":132,"threads_started":1}
I20260809 02:10:36.004110 16264 heartbeater.cc:347] Connected to a master server at 127.5.9.190:33497
I20260809 02:10:36.004241 16264 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:36.004449 16264 heartbeater.cc:511] Master 127.5.9.190:33497 requested a full tablet report, sending...
I20260809 02:10:36.004987 15930 ts_manager.cc:194] Registered new tserver with Master: af84e1b7696d49aa9b0b3090c86aa7c9 (127.5.9.130:39835)
I20260809 02:10:36.005407 15930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:43091
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:10:36.033041 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:33497
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:45057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:10:36.118870 16269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:10:36.119168 16269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:10:36.119213 16269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:10:36.120712 16269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:10:36.120808 16269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:10:36.120862 16269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:10:36.122896 16269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:45057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:33497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:10:36.123234 16269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:10:36.123513 16269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:10:36.128925 16274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:36.129029 16277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:10:36.128990 16275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:10:36.129194 16269 server_base.cc:1034] running on GCE node
I20260809 02:10:36.129387 16269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:10:36.129602 16269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:10:36.130833 16269 hybrid_clock.cc:648] HybridClock initialized: now 1786241436130806 us; error 45 us; skew 500 ppm
Aug 09 02:10:36 dist-test-slave-p3t5 krb5kdc[15883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241436, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:10:36.133496 16269 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:10:36.134024 16269 webserver.cc:460] Webserver started at http://127.5.9.131:37175/ using document root <none> and password file <none>
I20260809 02:10:36.134238 16269 fs_manager.cc:359] Metadata directory not provided
I20260809 02:10:36.134281 16269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:10:36.134377 16269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:10:36.135478 16269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "d09db342d2a2412b8664f0ce0f207c84"
format_stamp: "Formatted at 2026-08-09 02:10:36 on dist-test-slave-p3t5"
server_key: "cba6b92655c5a9d272ef75e87ced66bf"
server_key_iv: "ad957cbd067db1d90a8ad82beb42e041"
server_key_version: "encryptionkey@0"
I20260809 02:10:36.135805 16269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "d09db342d2a2412b8664f0ce0f207c84"
format_stamp: "Formatted at 2026-08-09 02:10:36 on dist-test-slave-p3t5"
server_key: "cba6b92655c5a9d272ef75e87ced66bf"
server_key_iv: "ad957cbd067db1d90a8ad82beb42e041"
server_key_version: "encryptionkey@0"
I20260809 02:10:36.137224 16269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:10:36.138051 16284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:36.138252 16269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:10:36.138345 16269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "d09db342d2a2412b8664f0ce0f207c84"
format_stamp: "Formatted at 2026-08-09 02:10:36 on dist-test-slave-p3t5"
server_key: "cba6b92655c5a9d272ef75e87ced66bf"
server_key_iv: "ad957cbd067db1d90a8ad82beb42e041"
server_key_version: "encryptionkey@0"
I20260809 02:10:36.138410 16269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:10:36.151389 16269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:10:36.153995 16269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:10:36.154116 16269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:10:36.154314 16269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:10:36.154661 16269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:10:36.154699 16269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:36.154747 16269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:10:36.154773 16269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:10:36.161605 16269 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:39753
I20260809 02:10:36.161657 16405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:39753 every 8 connection(s)
I20260809 02:10:36.162227 16269 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
Aug 09 02:10:36 dist-test-slave-p3t5 krb5kdc[15883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241436, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:10:36.169545 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 16269
I20260809 02:10:36.169700 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:10:36.169922 5158 external_mini_cluster.cc:1428] Setting key e18c930c7fef83f858c55fc256c74c95
I20260809 02:10:36.172546 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:36.163086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:59699 (local address 127.5.9.190:33497)
0809 02:10:36.163183 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:36.163185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:36.163393 (+ 208us) server_negotiation.cc:407] Connection header received
0809 02:10:36.164423 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:36.164424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:36.164437 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:36.164520 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:36.165206 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.165841 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.166849 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.167042 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.167104 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:36.169374 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:36.169375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:36.169376 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:36.169383 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:36.171246 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:36.171636 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:36.171637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:36.171637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:36.171695 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:36.171930 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:36.171931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:36.171931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:36.172041 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:36.172095 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:36.172400 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 02:10:36.172477 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 02:10:36.172789 16408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:36.163064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:33497 (local address 127.5.9.131:59699)
0809 02:10:36.163252 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:36.163304 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:36.164297 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:36.164628 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:36.164632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:36.164906 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:10:36.165108 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:36.165112 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.165929 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.165930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:36.166760 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:36.166762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.167123 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.167124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:36.167224 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:36.167826 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:10:36.167834 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:10:36.169284 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:10:36.171324 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:36.171330 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:36.171332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:36.171567 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:36.171758 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:10:36.171758 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:10:36.171759 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:10:36.171864 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:10:36.172099 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:10:36.172104 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:10:36.172225 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 02:10:36.172379 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 02:10:36.172514 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260809 02:10:36.173314 16406 heartbeater.cc:347] Connected to a master server at 127.5.9.190:33497
I20260809 02:10:36.173437 16406 heartbeater.cc:464] Registering TS with master...
I20260809 02:10:36.173607 16406 heartbeater.cc:511] Master 127.5.9.190:33497 requested a full tablet report, sending...
I20260809 02:10:36.174031 15930 ts_manager.cc:194] Registered new tserver with Master: d09db342d2a2412b8664f0ce0f207c84 (127.5.9.131:39753)
I20260809 02:10:36.174381 15930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:59699
I20260809 02:10:36.181886 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:10:36.189757 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:36.183317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41638 (local address 127.5.9.190:33497)
0809 02:10:36.183372 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:36.183374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:36.183440 (+ 66us) server_negotiation.cc:407] Connection header received
0809 02:10:36.183529 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:36.183530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:36.183541 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:36.183629 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:10:36.184257 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.184766 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.185525 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.185793 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.185873 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:36.187115 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:10:36.187116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:10:36.187117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:10:36.187125 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:10:36.188425 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:36.188822 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:36.188824 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:36.188824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:36.188895 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:10:36.189118 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:10:36.189119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:10:36.189120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:10:36.189294 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:10:36.189414 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:36.189537 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 02:10:36.189647 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 02:10:36.191421 15930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:10:36.193212 15930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:10:36.200100 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:36.196005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:39835 (local address 127.0.0.1:44182)
0809 02:10:36.196303 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:36.196312 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:36.196352 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:36.196867 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:36.196868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:36.196869 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:36.196901 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:36.196902 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.198111 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.198112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:36.199026 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:36.199029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.199766 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.199767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:36.199849 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:36.199863 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 02:10:36.199957 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 02:10:36.200017 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":74,"threads_started":1}
I20260809 02:10:36.200117 16422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:36.196253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44182 (local address 127.5.9.130:39835)
0809 02:10:36.196529 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:36.196534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:36.196543 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:10:36.196768 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:36.196771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:36.196790 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:36.196872 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:36.196969 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.198014 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.199169 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.199675 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.199766 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:36.199830 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:36.199987 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 02:10:36.200029 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":98,"threads_started":1}
I20260809 02:10:36.200421 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:36.196000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51102 (local address 127.5.9.131:39753)
0809 02:10:36.196213 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:36.196217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:36.196229 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:10:36.196277 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:36.196278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:36.196291 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:36.196383 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:36.196608 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.197808 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.199179 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.199913 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.199965 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:36.200021 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:36.200278 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 02:10:36.200326 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":92,"threads_started":1}
I20260809 02:10:36.200553 16419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:36.195952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48682 (local address 127.5.9.129:42871)
0809 02:10:36.196485 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:36.196489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:36.196500 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:10:36.196549 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:36.196551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:36.196567 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:36.196641 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:36.196779 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.197540 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.199193 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.199913 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.199984 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:36.200032 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:36.200425 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 02:10:36.200473 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":93,"threads_started":1}
I20260809 02:10:36.200627 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:36.195786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:39753 (local address 127.0.0.1:51102)
0809 02:10:36.196137 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:36.196148 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:36.196195 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:36.196412 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:36.196415 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:36.196418 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:36.196504 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:36.196506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.197912 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.197913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:36.199048 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:36.199051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.200111 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.200111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:36.200167 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:36.200171 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:10:36.200505 (+ 334us) client_negotiation.cc:240] Negotiation successful
0809 02:10:36.200550 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":4,"spinlock_wait_cycles":768,"thread_start_us":92,"threads_started":1}
I20260809 02:10:36.200349 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:36.195785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:42871 (local address 127.0.0.1:48682)
0809 02:10:36.195946 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:36.195993 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:36.196070 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:36.196648 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:36.196649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:36.196651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:36.196704 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:36.196705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.197674 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.197675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:36.199101 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:36.199103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.200070 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.200071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:36.200174 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:36.200177 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:10:36.200224 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 02:10:36.200271 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":41,"thread_start_us":72,"threads_started":1}
I20260809 02:10:36.201280 16194 tablet_service.cc:1467] Processing CreateTablet for tablet ceee947fb4e34a7b9a914ab559793063 (DEFAULT_TABLE table=test-table [id=0b55e07ee9d7498fbc4baaa8987d0e2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:36.201419 16335 tablet_service.cc:1467] Processing CreateTablet for tablet ceee947fb4e34a7b9a914ab559793063 (DEFAULT_TABLE table=test-table [id=0b55e07ee9d7498fbc4baaa8987d0e2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:36.201714 16194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceee947fb4e34a7b9a914ab559793063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:36.201864 16335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceee947fb4e34a7b9a914ab559793063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:36.204643 16423 tablet_bootstrap.cc:492] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9: Bootstrap starting.
I20260809 02:10:36.204659 16424 tablet_bootstrap.cc:492] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84: Bootstrap starting.
I20260809 02:10:36.205363 16424 tablet_bootstrap.cc:654] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:36.205432 16423 tablet_bootstrap.cc:654] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:36.205809 16059 tablet_service.cc:1467] Processing CreateTablet for tablet ceee947fb4e34a7b9a914ab559793063 (DEFAULT_TABLE table=test-table [id=0b55e07ee9d7498fbc4baaa8987d0e2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:10:36.205948 16424 log.cc:824] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:36.206077 16423 log.cc:824] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:36.206197 16059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceee947fb4e34a7b9a914ab559793063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:10:36.206910 16423 tablet_bootstrap.cc:492] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9: No bootstrap required, opened a new log
I20260809 02:10:36.206960 16423 ts_tablet_manager.cc:1397] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 02:10:36.208519 16423 raft_consensus.cc:348] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d09db342d2a2412b8664f0ce0f207c84" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39753 } } peers { permanent_uuid: "a2ae2c39a294405da6be1d243b474ebf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 42871 } } peers { permanent_uuid: "af84e1b7696d49aa9b0b3090c86aa7c9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39835 } }
I20260809 02:10:36.208647 16423 raft_consensus.cc:374] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:36.208667 16423 raft_consensus.cc:729] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af84e1b7696d49aa9b0b3090c86aa7c9, State: Initialized, Role: FOLLOWER
I20260809 02:10:36.208765 16423 consensus_queue.cc:260] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d09db342d2a2412b8664f0ce0f207c84" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39753 } } peers { permanent_uuid: "a2ae2c39a294405da6be1d243b474ebf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 42871 } } peers { permanent_uuid: "af84e1b7696d49aa9b0b3090c86aa7c9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39835 } }
I20260809 02:10:36.209036 16427 tablet_bootstrap.cc:492] T ceee947fb4e34a7b9a914ab559793063 P a2ae2c39a294405da6be1d243b474ebf: Bootstrap starting.
I20260809 02:10:36.209080 16423 ts_tablet_manager.cc:1428] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 02:10:36.209241 16264 heartbeater.cc:503] Master 127.5.9.190:33497 was elected leader, sending a full tablet report...
I20260809 02:10:36.209782 16427 tablet_bootstrap.cc:654] T ceee947fb4e34a7b9a914ab559793063 P a2ae2c39a294405da6be1d243b474ebf: Neither blocks nor log segments found. Creating new log.
I20260809 02:10:36.209808 16424 tablet_bootstrap.cc:492] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84: No bootstrap required, opened a new log
I20260809 02:10:36.209874 16424 ts_tablet_manager.cc:1397] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 02:10:36.210419 16427 log.cc:824] T ceee947fb4e34a7b9a914ab559793063 P a2ae2c39a294405da6be1d243b474ebf: Log is configured to *not* fsync() on all Append() calls
I20260809 02:10:36.211269 16427 tablet_bootstrap.cc:492] T ceee947fb4e34a7b9a914ab559793063 P a2ae2c39a294405da6be1d243b474ebf: No bootstrap required, opened a new log
I20260809 02:10:36.211359 16427 ts_tablet_manager.cc:1397] T ceee947fb4e34a7b9a914ab559793063 P a2ae2c39a294405da6be1d243b474ebf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 02:10:36.211728 16424 raft_consensus.cc:348] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d09db342d2a2412b8664f0ce0f207c84" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39753 } } peers { permanent_uuid: "a2ae2c39a294405da6be1d243b474ebf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 42871 } } peers { permanent_uuid: "af84e1b7696d49aa9b0b3090c86aa7c9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39835 } }
I20260809 02:10:36.211933 16424 raft_consensus.cc:374] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:36.212020 16424 raft_consensus.cc:729] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d09db342d2a2412b8664f0ce0f207c84, State: Initialized, Role: FOLLOWER
I20260809 02:10:36.212173 16424 consensus_queue.cc:260] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d09db342d2a2412b8664f0ce0f207c84" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39753 } } peers { permanent_uuid: "a2ae2c39a294405da6be1d243b474ebf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 42871 } } peers { permanent_uuid: "af84e1b7696d49aa9b0b3090c86aa7c9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39835 } }
I20260809 02:10:36.212478 16424 ts_tablet_manager.cc:1428] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:10:36.212553 16406 heartbeater.cc:503] Master 127.5.9.190:33497 was elected leader, sending a full tablet report...
I20260809 02:10:36.213330 16427 raft_consensus.cc:348] T ceee947fb4e34a7b9a914ab559793063 P a2ae2c39a294405da6be1d243b474ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d09db342d2a2412b8664f0ce0f207c84" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39753 } } peers { permanent_uuid: "a2ae2c39a294405da6be1d243b474ebf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 42871 } } peers { permanent_uuid: "af84e1b7696d49aa9b0b3090c86aa7c9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39835 } }
I20260809 02:10:36.213497 16427 raft_consensus.cc:374] T ceee947fb4e34a7b9a914ab559793063 P a2ae2c39a294405da6be1d243b474ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:10:36.213534 16427 raft_consensus.cc:729] T ceee947fb4e34a7b9a914ab559793063 P a2ae2c39a294405da6be1d243b474ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ae2c39a294405da6be1d243b474ebf, State: Initialized, Role: FOLLOWER
I20260809 02:10:36.213743 16427 consensus_queue.cc:260] T ceee947fb4e34a7b9a914ab559793063 P a2ae2c39a294405da6be1d243b474ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d09db342d2a2412b8664f0ce0f207c84" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39753 } } peers { permanent_uuid: "a2ae2c39a294405da6be1d243b474ebf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 42871 } } peers { permanent_uuid: "af84e1b7696d49aa9b0b3090c86aa7c9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39835 } }
I20260809 02:10:36.214097 16427 ts_tablet_manager.cc:1428] T ceee947fb4e34a7b9a914ab559793063 P a2ae2c39a294405da6be1d243b474ebf: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 02:10:36.214112 16125 heartbeater.cc:503] Master 127.5.9.190:33497 was elected leader, sending a full tablet report...
I20260809 02:10:36.234176 16428 raft_consensus.cc:482] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:10:36.234313 16428 raft_consensus.cc:504] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d09db342d2a2412b8664f0ce0f207c84" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39753 } } peers { permanent_uuid: "a2ae2c39a294405da6be1d243b474ebf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 42871 } } peers { permanent_uuid: "af84e1b7696d49aa9b0b3090c86aa7c9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39835 } }
I20260809 02:10:36.234679 16428 leader_election.cc:288] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d09db342d2a2412b8664f0ce0f207c84 (127.5.9.131:39753), a2ae2c39a294405da6be1d243b474ebf (127.5.9.129:42871)
I20260809 02:10:36.238876 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:36.235014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:47491 (local address 127.5.9.131:39753)
0809 02:10:36.235064 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:36.235066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:36.235079 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:10:36.235138 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:36.235139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:36.235152 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:36.235237 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:36.235451 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.236234 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.237842 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.238461 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.238531 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:36.238542 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:36.238748 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 02:10:36.238796 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:10:36.238876 16419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:36.235040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:56349 (local address 127.5.9.129:42871)
0809 02:10:36.235119 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:36.235121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:36.235215 (+ 94us) server_negotiation.cc:407] Connection header received
0809 02:10:36.235270 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:36.235272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:36.235284 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:36.235342 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:10:36.235519 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.236224 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.237842 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.238441 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.238530 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:36.238542 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:36.238749 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 02:10:36.238796 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:10:36.238909 16266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:36.234902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:39753 (local address 127.5.9.130:47491)
0809 02:10:36.234991 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:36.235008 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:36.235070 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:36.235263 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:36.235264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:36.235266 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:36.235373 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:36.235375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.236325 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.236326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:36.237713 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:36.237717 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.238555 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.238557 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:36.238650 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:36.238657 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 02:10:36.238723 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 02:10:36.238777 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":78}
I20260809 02:10:36.238915 16433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:36.234902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:42871 (local address 127.5.9.130:56349)
0809 02:10:36.235127 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:36.235137 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:10:36.235194 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:10:36.235367 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:10:36.235368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:10:36.235369 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:10:36.235430 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:36.235431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.236325 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.236326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:36.237713 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:10:36.237717 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.238556 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.238557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:10:36.238650 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:36.238658 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 02:10:36.238723 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 02:10:36.238777 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":95,"thread_start_us":130,"threads_started":1}
I20260809 02:10:36.239439 16356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceee947fb4e34a7b9a914ab559793063" candidate_uuid: "af84e1b7696d49aa9b0b3090c86aa7c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d09db342d2a2412b8664f0ce0f207c84" is_pre_election: true
I20260809 02:10:36.239650 16356 raft_consensus.cc:2454] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af84e1b7696d49aa9b0b3090c86aa7c9 in term 0.
I20260809 02:10:36.239675 16079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceee947fb4e34a7b9a914ab559793063" candidate_uuid: "af84e1b7696d49aa9b0b3090c86aa7c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ae2c39a294405da6be1d243b474ebf" is_pre_election: true
I20260809 02:10:36.239874 16079 raft_consensus.cc:2454] T ceee947fb4e34a7b9a914ab559793063 P a2ae2c39a294405da6be1d243b474ebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af84e1b7696d49aa9b0b3090c86aa7c9 in term 0.
I20260809 02:10:36.239974 16146 leader_election.cc:302] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af84e1b7696d49aa9b0b3090c86aa7c9, d09db342d2a2412b8664f0ce0f207c84; no voters:
I20260809 02:10:36.240136 16428 raft_consensus.cc:2781] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:10:36.240207 16428 raft_consensus.cc:482] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:10:36.240226 16428 raft_consensus.cc:3037] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:36.241080 16428 raft_consensus.cc:504] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d09db342d2a2412b8664f0ce0f207c84" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39753 } } peers { permanent_uuid: "a2ae2c39a294405da6be1d243b474ebf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 42871 } } peers { permanent_uuid: "af84e1b7696d49aa9b0b3090c86aa7c9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39835 } }
I20260809 02:10:36.241246 16428 leader_election.cc:288] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [CANDIDATE]: Term 1 election: Requested vote from peers d09db342d2a2412b8664f0ce0f207c84 (127.5.9.131:39753), a2ae2c39a294405da6be1d243b474ebf (127.5.9.129:42871)
I20260809 02:10:36.241442 16079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceee947fb4e34a7b9a914ab559793063" candidate_uuid: "af84e1b7696d49aa9b0b3090c86aa7c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ae2c39a294405da6be1d243b474ebf"
I20260809 02:10:36.241465 16356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceee947fb4e34a7b9a914ab559793063" candidate_uuid: "af84e1b7696d49aa9b0b3090c86aa7c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d09db342d2a2412b8664f0ce0f207c84"
I20260809 02:10:36.241545 16356 raft_consensus.cc:3037] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:36.241552 16079 raft_consensus.cc:3037] T ceee947fb4e34a7b9a914ab559793063 P a2ae2c39a294405da6be1d243b474ebf [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:10:36.242465 16079 raft_consensus.cc:2454] T ceee947fb4e34a7b9a914ab559793063 P a2ae2c39a294405da6be1d243b474ebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af84e1b7696d49aa9b0b3090c86aa7c9 in term 1.
I20260809 02:10:36.242482 16356 raft_consensus.cc:2454] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af84e1b7696d49aa9b0b3090c86aa7c9 in term 1.
I20260809 02:10:36.242679 16146 leader_election.cc:302] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a2ae2c39a294405da6be1d243b474ebf, af84e1b7696d49aa9b0b3090c86aa7c9, d09db342d2a2412b8664f0ce0f207c84; no voters:
I20260809 02:10:36.242818 16428 raft_consensus.cc:2781] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:10:36.243057 16428 raft_consensus.cc:686] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [term 1 LEADER]: Becoming Leader. State: Replica: af84e1b7696d49aa9b0b3090c86aa7c9, State: Running, Role: LEADER
W20260809 02:10:36.243161 16265 tablet.cc:2369] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:10:36.243178 16428 consensus_queue.cc:237] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d09db342d2a2412b8664f0ce0f207c84" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39753 } } peers { permanent_uuid: "a2ae2c39a294405da6be1d243b474ebf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 42871 } } peers { permanent_uuid: "af84e1b7696d49aa9b0b3090c86aa7c9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39835 } }
I20260809 02:10:36.244087 15930 catalog_manager.cc:5495] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to af84e1b7696d49aa9b0b3090c86aa7c9 (127.5.9.130). New cstate: current_term: 1 leader_uuid: "af84e1b7696d49aa9b0b3090c86aa7c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d09db342d2a2412b8664f0ce0f207c84" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2ae2c39a294405da6be1d243b474ebf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 42871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af84e1b7696d49aa9b0b3090c86aa7c9" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39835 } health_report { overall_health: HEALTHY } } }
I20260809 02:10:36.258878 16422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:36.256296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44184 (local address 127.5.9.130:39835)
0809 02:10:36.256365 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 02:10:36.256367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:10:36.256376 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:10:36.256439 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:10:36.256440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:10:36.256464 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:10:36.256526 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:10:36.256659 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.257218 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.258113 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:10:36.258300 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:10:36.258355 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:10:36.258506 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:10:36.258610 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:10:36.258672 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:10:36.258704 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 02:10:36.258724 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:10:36.260907 16356 raft_consensus.cc:1261] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84 [term 1 FOLLOWER]: Refusing update from remote peer af84e1b7696d49aa9b0b3090c86aa7c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:36.260933 16079 raft_consensus.cc:1261] T ceee947fb4e34a7b9a914ab559793063 P a2ae2c39a294405da6be1d243b474ebf [term 1 FOLLOWER]: Refusing update from remote peer af84e1b7696d49aa9b0b3090c86aa7c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:10:36.261332 16434 consensus_queue.cc:1035] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ae2c39a294405da6be1d243b474ebf" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 42871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:36.261498 16428 consensus_queue.cc:1035] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d09db342d2a2412b8664f0ce0f207c84" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:10:36.264364 16440 mvcc.cc:205] Tried to move back new op lower bound from 7316444922921291776 to 7316444922852646912. Current Snapshot: MvccSnapshot[applied={T|T < 7316444922921291776}]
I20260809 02:10:36.264514 16438 mvcc.cc:205] Tried to move back new op lower bound from 7316444922921291776 to 7316444922852646912. Current Snapshot: MvccSnapshot[applied={T|T < 7316444922921291776 or (T in {7316444922921291776})}]
I20260809 02:10:36.264901 16439 mvcc.cc:205] Tried to move back new op lower bound from 7316444922921291776 to 7316444922852646912. Current Snapshot: MvccSnapshot[applied={T|T < 7316444922921291776}]
I20260809 02:10:36.266852 15930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:36.266944 15930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:10:36.267818 15930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e2ae3ba763443f3abd170c949e7f50e: Sending DeleteTablet for 3 replicas of tablet ceee947fb4e34a7b9a914ab559793063
I20260809 02:10:36.268200 16059 tablet_service.cc:1514] Processing DeleteTablet for tablet ceee947fb4e34a7b9a914ab559793063 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:36 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:48682
I20260809 02:10:36.268200 16335 tablet_service.cc:1514] Processing DeleteTablet for tablet ceee947fb4e34a7b9a914ab559793063 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:36 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:51102
I20260809 02:10:36.268491 16194 tablet_service.cc:1514] Processing DeleteTablet for tablet ceee947fb4e34a7b9a914ab559793063 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:10:36 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:44182
I20260809 02:10:36.268716 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 15994
I20260809 02:10:36.268783 16447 tablet_replica.cc:330] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84: stopping tablet replica
I20260809 02:10:36.268801 16449 tablet_replica.cc:330] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9: stopping tablet replica
I20260809 02:10:36.268901 16449 raft_consensus.cc:2229] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:10:36.269030 16449 raft_consensus.cc:2258] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:36.270257 16447 raft_consensus.cc:2229] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:10:36.270488 16447 raft_consensus.cc:2258] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:10:36.271374 16447 ts_tablet_manager.cc:1905] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:36.272336 16449 ts_tablet_manager.cc:1905] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:10:36.274425 16447 ts_tablet_manager.cc:1918] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:36.274466 16449 ts_tablet_manager.cc:1918] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:10:36.274542 16449 log.cc:1196] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/ceee947fb4e34a7b9a914ab559793063
I20260809 02:10:36.274662 16447 log.cc:1196] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/ceee947fb4e34a7b9a914ab559793063
I20260809 02:10:36.274977 16449 ts_tablet_manager.cc:1939] T ceee947fb4e34a7b9a914ab559793063 P af84e1b7696d49aa9b0b3090c86aa7c9: Deleting consensus metadata
I20260809 02:10:36.275170 16447 ts_tablet_manager.cc:1939] T ceee947fb4e34a7b9a914ab559793063 P d09db342d2a2412b8664f0ce0f207c84: Deleting consensus metadata
I20260809 02:10:36.275563 15914 catalog_manager.cc:4841] TS af84e1b7696d49aa9b0b3090c86aa7c9 (127.5.9.130:39835): tablet ceee947fb4e34a7b9a914ab559793063 (table test-table [id=0b55e07ee9d7498fbc4baaa8987d0e2f]) successfully deleted
I20260809 02:10:36.275784 15914 catalog_manager.cc:4841] TS d09db342d2a2412b8664f0ce0f207c84 (127.5.9.131:39753): tablet ceee947fb4e34a7b9a914ab559793063 (table test-table [id=0b55e07ee9d7498fbc4baaa8987d0e2f]) successfully deleted
W20260809 02:10:36.282940 15917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:42871 (error 108)
I20260809 02:10:36.283633 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:10:36.283470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:42871 (local address 127.0.0.1:48684)
0809 02:10:36.283541 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 02:10:36.283589 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 02:10:36.283684 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 16130
W20260809 02:10:36.285815 15917 catalog_manager.cc:4584] TS a2ae2c39a294405da6be1d243b474ebf (127.5.9.129:42871): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ceee947fb4e34a7b9a914ab559793063: Network error: Client connection negotiation failed: client connection to 127.5.9.129:42871: connect: Connection refused (error 111)
I20260809 02:10:36.290709 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 16269
I20260809 02:10:36.297195 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 15897
2026-08-09T02:10:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2044 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:10:36 dist-test-slave-p3t5 krb5kdc[16453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:10:36 dist-test-slave-p3t5 krb5kdc[16453](info): set up 2 sockets
Aug 09 02:10:36 dist-test-slave-p3t5 krb5kdc[16453](info): commencing operation
krb5kdc: starting...
W20260809 02:10:37.403472 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:10:37 dist-test-slave-p3t5 krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:10:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:10:37Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.9.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.9.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.9.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/rangeradmin_127.5.9.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.9.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/rangeradmin_127.5.9.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.9.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.9.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.9.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/rangerlookup_127.5.9.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.9.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/rangerlookup_127.5.9.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.9.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.9.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.9.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/HTTP_127.5.9.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.9.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/HTTP_127.5.9.148@KRBTEST.COM.keytab.
I20260809 02:10:37.590083 5158 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasktks196/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/postgres -l logfile start
2026-08-09 02:10:40.410 UTC [16483] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 02:10:40.411 UTC [16483] LOG: listening on IPv4 address "127.5.9.148", port 52511
2026-08-09 02:10:40.415 UTC [16483] LOG: listening on Unix socket "/tmp/.s.PGSQL.52511"
2026-08-09 02:10:40.439 UTC [16486] LOG: database system was shut down at 2026-08-09 02:10:38 UTC
2026-08-09 02:10:40.443 UTC [16483] LOG: database system is ready to accept connections
I20260809 02:10:41.423259 5158 mini_postgres.cc:96] Postgres bound to 52511
2026-08-09 02:10:41.432 UTC [16495] FATAL: database "slave" does not exist
127.5.9.148:52511 - accepting connections
I20260809 02:10:41.433887 5158 mini_ranger.cc:161] Starting Ranger...
I20260809 02:10:41.453003 5158 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 02:10:42.006688 5158 mini_ranger.cc:87] Created ranger Postgres database
I20260809 02:10:42.006853 5158 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-admin
2026-08-09 02:10:42,329 [I] DB FLAVOR :POSTGRES
2026-08-09 02:10:42,330 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 02:10:42,330 [I] Checking connection..
2026-08-09 02:10:42,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 02:10:42,605 [I] Checking connection passed.
2026-08-09 02:10:42,606 [I] --------- Verifying version history table ---------
2026-08-09 02:10:42,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:10:42,878 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 02:10:42,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:10:43,131 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 02:10:43,132 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 02:10:43,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 02:10:43.399 UTC [16562] WARNING: there is no transaction in progress
2026-08-09 02:10:43,407 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 02:10:43,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:10:43,678 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 02:10:43,679 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 02:10:43,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 02:10:43,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 02:10:44,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-p3t5.c.gcp-upstream.internal','N') ;"
2026-08-09 02:10:44,453 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 02:10:44,714 [I] Table x_portal_user does not exist in database ranger
2026-08-09 02:10:44,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 02:10:44,994 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 02:10:44,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 02:10:45,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 02:10:45,551 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 02:10:45,551 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 02:10:46.595 UTC [16675] WARNING: there is no transaction in progress
2026-08-09 02:10:46.613 UTC [16675] WARNING: there is no transaction in progress
2026-08-09 02:10:46.632 UTC [16675] WARNING: there is no transaction in progress
2026-08-09 02:10:46.648 UTC [16675] WARNING: there is no transaction in progress
2026-08-09 02:10:46.667 UTC [16675] WARNING: there is no transaction in progress
2026-08-09 02:10:46.685 UTC [16675] WARNING: there is no transaction in progress
2026-08-09 02:10:46.745 UTC [16675] WARNING: there is no transaction in progress
2026-08-09 02:10:46.753 UTC [16675] WARNING: there is no transaction in progress
2026-08-09 02:10:46.771 UTC [16675] WARNING: there is no transaction in progress
2026-08-09 02:10:46.783 UTC [16675] WARNING: there is no transaction in progress
2026-08-09 02:10:46.795 UTC [16675] WARNING: there is no transaction in progress
2026-08-09 02:10:46.810 UTC [16675] WARNING: there is no transaction in progress
2026-08-09 02:10:46.820 UTC [16675] WARNING: there is no transaction in progress
2026-08-09 02:10:46.827 UTC [16675] WARNING: there is no transaction in progress
2026-08-09 02:10:46.835 UTC [16675] WARNING: there is no transaction in progress
2026-08-09 02:10:46.843 UTC [16675] WARNING: there is no transaction in progress
2026-08-09 02:10:47,580 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 02:10:47,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 02:10:47,857 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 02:10:47,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 02:10:48,130 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 02:10:48,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 02:10:48,382 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 02:10:48,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 02:10:48,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 02:10:48,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-p3t5.c.gcp-upstream.internal';"
2026-08-09 02:10:49,180 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 02:10:49,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 02:10:49,446 [I] DB_PATCHES have already been applied
I20260809 02:10:49.452383 5158 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-admin:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasktks196/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 02:10:49.452466 5158 mini_ranger.cc:193] Using host: 127.5.9.148
I20260809 02:10:49.456485 5158 mini_ranger.cc:227] Ranger admin URL: http://127.5.9.148:42423
Aug 09, 2026 2:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 2:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 2:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 2:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 2:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 2:10:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 2:10:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 02:10:50.615218 5158 mini_ranger.cc:160] Time spent starting Ranger: real 9.181s user 0.000s sys 0.012s
Aug 09, 2026 2:11:12 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 2:11:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 2:11:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 2:11:12 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 2:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 2:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 2:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 2:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 2:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 2:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 2:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 2:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 2:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 2:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 2:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 2:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 2:11:12 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 2:11:13 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 02:11:14.025007 5158 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.9.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.9.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.9.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/rangerkms_127.5.9.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.9.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/rangerkms_127.5.9.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.9.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.9.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.9.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/HTTP_127.5.9.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.9.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/HTTP_127.5.9.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 02:11:14 dist-test-slave-p3t5 krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241474, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 02:11:14.118101 5158 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 02:11:14.128988 5158 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 02:11:15.092990 5158 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 02:11:15.093088 5158 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-kms
I20260809 02:11:15.093111 5158 mini_ranger_kms.cc:227] Using postgres at 127.5.9.148:52511
2026-08-09 02:11:15,474 [I] DB FLAVOR :POSTGRES
2026-08-09 02:11:15,474 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 02:11:15,474 [I] Checking connection
2026-08-09 02:11:15,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 02:11:15,770 [I] connection success
2026-08-09 02:11:15,770 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 02:11:15,770 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 02:11:15,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 02:11:16,031 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 02:11:16,032 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 02:11:16,032 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 02:11:16,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:52511/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 02:11:16,339 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 02:11:16.909492 5158 mini_ranger_kms.cc:310] Created kms service
I20260809 02:11:17.121675 5158 mini_ranger_kms.cc:326] Created kudu user
I20260809 02:11:17.171936 5158 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 02:11:17 dist-test-slave-p3t5 krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241474, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
I20260809 02:11:17.622120 5158 mini_ranger_kms.cc:384] Added ranger policy
I20260809 02:11:17.622251 5158 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-kms:/tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 02:11:17.622292 5158 mini_ranger_kms.cc:240] Using host: 127.5.9.148
I20260809 02:11:17.626768 5158 mini_ranger_kms.cc:276] Ranger KMS PID: 16863
I20260809 02:11:17.626926 5158 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.9.148:49005
Aug 09, 2026 2:11:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 2:11:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 2:11:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 2:11:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 2:11:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 2:11:19 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 02:11:19.943701 5158 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.826s user 0.001s sys 0.359s
I20260809 02:11:19.943827 5158 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 02:11:19.943909 5158 mini_ranger_kms.cc:401] 127.5.9.148:49005/kms/v1/keys
Aug 09 02:11:20 dist-test-slave-p3t5 krb5kdc[16453](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241480, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.9.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 02:11:20 dist-test-slave-p3t5 krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241480, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.9.148@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
Aug 09 02:11:21 dist-test-slave-p3t5 krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241480, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.9.148@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
Aug 09 02:11:23 dist-test-slave-p3t5 krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241483, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:11:23.517550 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:37151
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:37151
--ranger_config_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.9.148:49005/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:11:23.620959 16914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:11:23.621203 16914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:11:23.621228 16914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:11:23.622712 16914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:11:23.622769 16914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:11:23.622784 16914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:11:23.622797 16914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:11:23.622809 16914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:11:23.625398 16914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.9.148:49005/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:37151
--ranger_config_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:37151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:11:23.625717 16914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:11:23.626000 16914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:11:23.632246 16919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:11:23.632200 16920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:11:23.632172 16922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:11:23.632484 16914 server_base.cc:1034] running on GCE node
I20260809 02:11:23.632876 16914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:11:23.633309 16914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:11:23.634498 16914 hybrid_clock.cc:648] HybridClock initialized: now 1786241483634473 us; error 42 us; skew 500 ppm
Aug 09 02:11:23 dist-test-slave-p3t5 krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241483, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:11:23.637688 16914 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:11:23.638306 16914 webserver.cc:460] Webserver started at http://127.5.9.190:39263/ using document root <none> and password file <none>
I20260809 02:11:23.638578 16914 fs_manager.cc:359] Metadata directory not provided
I20260809 02:11:23.638640 16914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:11:23.638810 16914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:11:23 dist-test-slave-p3t5 krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241483, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
I20260809 02:11:23.696904 16914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "945412525acb4fdabd8219e527c5cc5a"
format_stamp: "Formatted at 2026-08-09 02:11:23 on dist-test-slave-p3t5"
server_key: "c88c3b042bff0514805ee23f4085ec61"
server_key_iv: "2ec2f888e60fe0e1bf0a1fa362f5c723"
server_key_version: "kuduclusterkey@0"
I20260809 02:11:23.697363 16914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "945412525acb4fdabd8219e527c5cc5a"
format_stamp: "Formatted at 2026-08-09 02:11:23 on dist-test-slave-p3t5"
server_key: "c88c3b042bff0514805ee23f4085ec61"
server_key_iv: "2ec2f888e60fe0e1bf0a1fa362f5c723"
server_key_version: "kuduclusterkey@0"
I20260809 02:11:23.699143 16914 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 02:11:23.712580 16929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:11:23.712797 16914 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:11:23.712924 16914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "945412525acb4fdabd8219e527c5cc5a"
format_stamp: "Formatted at 2026-08-09 02:11:23 on dist-test-slave-p3t5"
server_key: "c88c3b042bff0514805ee23f4085ec61"
server_key_iv: "2ec2f888e60fe0e1bf0a1fa362f5c723"
server_key_version: "kuduclusterkey@0"
I20260809 02:11:23.713047 16914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:11:23.742831 16914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:11:23.745115 16914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:11:23.745314 16914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:11:23.751840 16914 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:37151
I20260809 02:11:23.751870 16983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:37151 every 8 connection(s)
I20260809 02:11:23.752348 16914 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:11:23.753203 16984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:11:23.756009 16984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a: Bootstrap starting.
I20260809 02:11:23.756729 16984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a: Neither blocks nor log segments found. Creating new log.
I20260809 02:11:23.757141 16984 log.cc:824] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:11:23.757507 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 16914
I20260809 02:11:23.757644 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:11:23.757846 5158 external_mini_cluster.cc:1428] Setting key e2a6112e01d52f3eaa74c8156aafc64b
I20260809 02:11:23.758206 16984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a: No bootstrap required, opened a new log
I20260809 02:11:23.761512 16984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945412525acb4fdabd8219e527c5cc5a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37151 } }
I20260809 02:11:23.761737 16984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:11:23.761785 16984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945412525acb4fdabd8219e527c5cc5a, State: Initialized, Role: FOLLOWER
I20260809 02:11:23.761920 16984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945412525acb4fdabd8219e527c5cc5a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37151 } }
I20260809 02:11:23.762013 16984 raft_consensus.cc:388] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:11:23.762079 16984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:11:23.762125 16984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:11:23.763352 16984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945412525acb4fdabd8219e527c5cc5a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37151 } }
I20260809 02:11:23.763491 16984 leader_election.cc:302] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 945412525acb4fdabd8219e527c5cc5a; no voters:
I20260809 02:11:23.763670 16984 leader_election.cc:288] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:11:23.763763 16989 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:11:23.763926 16989 raft_consensus.cc:686] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [term 1 LEADER]: Becoming Leader. State: Replica: 945412525acb4fdabd8219e527c5cc5a, State: Running, Role: LEADER
I20260809 02:11:23.764029 16984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:11:23.764457 16989 consensus_queue.cc:237] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945412525acb4fdabd8219e527c5cc5a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37151 } }
I20260809 02:11:23.766292 16990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 945412525acb4fdabd8219e527c5cc5a. Latest consensus state: current_term: 1 leader_uuid: "945412525acb4fdabd8219e527c5cc5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945412525acb4fdabd8219e527c5cc5a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37151 } } }
Aug 09 02:11:23 dist-test-slave-p3t5 krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241483, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:11:23.766398 16991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "945412525acb4fdabd8219e527c5cc5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945412525acb4fdabd8219e527c5cc5a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37151 } } }
I20260809 02:11:23.767093 16991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [sys.catalog]: This master's current role is: LEADER
I20260809 02:11:23.767351 16984 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 02:11:23.767665 16990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a [sys.catalog]: This master's current role is: LEADER
I20260809 02:11:23.767905 17005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:11:23.771715 17005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:11:23.775687 16988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:23.758463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36344 (local address 127.5.9.190:37151)
0809 02:11:23.758663 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 02:11:23.758673 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:11:23.758785 (+ 112us) server_negotiation.cc:407] Connection header received
0809 02:11:23.759818 (+ 1033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:11:23.759829 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:11:23.759896 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:11:23.760055 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:11:23.760755 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:23.762331 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:23.763308 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:23.764015 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:23.764154 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:23.767372 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:11:23.767377 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:11:23.767377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:11:23.767391 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:11:23.773204 (+ 5813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:11:23.773978 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:11:23.773982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:11:23.773983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:11:23.774168 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:11:23.774424 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:11:23.774425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:11:23.774425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:11:23.774927 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:11:23.775138 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 02:11:23.775214 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 02:11:23.775404 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
I20260809 02:11:23.778350 17005 catalog_manager.cc:1347] Generated new cluster ID: 78f4430acacd49b590dad15f4bdb4339
I20260809 02:11:23.778434 17005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:11:23.786764 17005 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:11:23.787344 17005 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:11:23.796960 17005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a: Generated new TSK 0
I20260809 02:11:23.797209 17005 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:11:24.616935 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.9.148:49005/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:37151
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:11:24.773586 17033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:11:24.774046 17033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:11:24.774096 17033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:11:24.776253 17033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:11:24.776343 17033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:11:24.776407 17033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:11:24.779285 17033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.9.148:49005/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:37151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:11:24.779618 17033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:11:24.780009 17033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:11:24.787055 17039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:11:24.787160 17042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:11:24.787055 17040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:11:24.788069 17033 server_base.cc:1034] running on GCE node
I20260809 02:11:24.788333 17033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:11:24.788650 17033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:11:24.789831 17033 hybrid_clock.cc:648] HybridClock initialized: now 1786241484789807 us; error 38 us; skew 500 ppm
Aug 09 02:11:24 dist-test-slave-p3t5 krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241484, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:11:24.793426 17033 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:11:24.794129 17033 webserver.cc:460] Webserver started at http://127.5.9.129:43347/ using document root <none> and password file <none>
I20260809 02:11:24.794420 17033 fs_manager.cc:359] Metadata directory not provided
I20260809 02:11:24.794513 17033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:11:24.794719 17033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:11:24 dist-test-slave-p3t5 krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241484, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
I20260809 02:11:24.810111 17033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "4da26eecacfb484b839ec0e4b558db2a"
format_stamp: "Formatted at 2026-08-09 02:11:24 on dist-test-slave-p3t5"
server_key: "0ea4ec82b3158fdcd98eb8660dd9c225"
server_key_iv: "a1f16283bea78a908e7c0164f9e2913d"
server_key_version: "kuduclusterkey@0"
I20260809 02:11:24.810659 17033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "4da26eecacfb484b839ec0e4b558db2a"
format_stamp: "Formatted at 2026-08-09 02:11:24 on dist-test-slave-p3t5"
server_key: "0ea4ec82b3158fdcd98eb8660dd9c225"
server_key_iv: "a1f16283bea78a908e7c0164f9e2913d"
server_key_version: "kuduclusterkey@0"
I20260809 02:11:24.816661 17033 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260809 02:11:24.832330 17050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:11:24.832916 17033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:11:24.833284 17033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "4da26eecacfb484b839ec0e4b558db2a"
format_stamp: "Formatted at 2026-08-09 02:11:24 on dist-test-slave-p3t5"
server_key: "0ea4ec82b3158fdcd98eb8660dd9c225"
server_key_iv: "a1f16283bea78a908e7c0164f9e2913d"
server_key_version: "kuduclusterkey@0"
I20260809 02:11:24.833527 17033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:11:24.849592 17033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:11:24.852072 17033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:11:24.852285 17033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:11:24.852626 17033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:11:24.853089 17033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:11:24.853155 17033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:11:24.853209 17033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:11:24.853238 17033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:11:24.866320 17033 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:36619
I20260809 02:11:24.866911 17172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:36619 every 8 connection(s)
I20260809 02:11:24.866997 17033 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:11:24.867549 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 17033
I20260809 02:11:24.867663 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:11:24.867879 5158 external_mini_cluster.cc:1428] Setting key 248ec6a8993fa5f6f3a4924c27f3e80f
Aug 09 02:11:24 dist-test-slave-p3t5 krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241484, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
I20260809 02:11:24.884152 17175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:24.869035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:50165 (local address 127.5.9.190:37151)
0809 02:11:24.870960 (+ 1925us) server_negotiation.cc:206] Beginning negotiation
0809 02:11:24.870963 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:11:24.871781 (+ 818us) server_negotiation.cc:407] Connection header received
0809 02:11:24.872649 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:11:24.872651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:11:24.872668 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:11:24.872768 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:11:24.873833 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:24.874692 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:24.876400 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:24.876745 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:24.876847 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:24.880156 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:11:24.880174 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:11:24.880175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:11:24.880186 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:11:24.882333 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:11:24.882913 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:11:24.882914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:11:24.882915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:11:24.882995 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:11:24.883316 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:11:24.883317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:11:24.883317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:11:24.883463 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:11:24.883549 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 02:11:24.883901 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 02:11:24.884028 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1656,"thread_start_us":104,"threads_started":1}
I20260809 02:11:24.884462 17176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:24.868880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:37151 (local address 127.5.9.129:50165)
0809 02:11:24.871608 (+ 2728us) negotiation.cc:107] Waiting for socket to connect
0809 02:11:24.871660 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:11:24.872517 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:11:24.872902 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:11:24.872907 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:11:24.873291 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:11:24.873714 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:11:24.873722 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:11:24.874830 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:11:24.874832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:11:24.876286 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:11:24.876290 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:11:24.876878 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:11:24.876879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:11:24.877056 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:24.878006 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:11:24.878020 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:11:24.879986 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:11:24.882462 (+ 2476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:11:24.882474 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:11:24.882478 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:11:24.882814 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:11:24.883083 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:11:24.883083 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:11:24.883084 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:11:24.883224 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:11:24.883541 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:11:24.883550 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:11:24.883739 (+ 189us) client_negotiation.cc:769] Sending connection context
0809 02:11:24.883864 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 02:11:24.884191 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2653,"thread_start_us":2329,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
I20260809 02:11:24.885452 17173 heartbeater.cc:347] Connected to a master server at 127.5.9.190:37151
I20260809 02:11:24.885571 17173 heartbeater.cc:464] Registering TS with master...
I20260809 02:11:24.885840 17173 heartbeater.cc:511] Master 127.5.9.190:37151 requested a full tablet report, sending...
I20260809 02:11:24.886420 16946 ts_manager.cc:194] Registered new tserver with Master: 4da26eecacfb484b839ec0e4b558db2a (127.5.9.129:36619)
I20260809 02:11:24.887099 16946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:50165
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:11:24.905028 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.9.148:49005/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:37151
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:11:25.052764 17179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:11:25.053144 17179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:11:25.053182 17179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:11:25.055227 17179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:11:25.055316 17179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:11:25.055367 17179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:11:25.058133 17179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.9.148:49005/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:37151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:11:25.058465 17179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:11:25.058842 17179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:11:25.066520 17187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:11:25.066545 17189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:11:25.067186 17179 server_base.cc:1034] running on GCE node
W20260809 02:11:25.067422 17186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:11:25.067675 17179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:11:25.067991 17179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:11:25.069186 17179 hybrid_clock.cc:648] HybridClock initialized: now 1786241485069147 us; error 51 us; skew 500 ppm
Aug 09 02:11:25 dist-test-slave-p3t5 krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241485, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:11:25.073101 17179 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:11:25.074219 17179 webserver.cc:460] Webserver started at http://127.5.9.130:36477/ using document root <none> and password file <none>
I20260809 02:11:25.074532 17179 fs_manager.cc:359] Metadata directory not provided
I20260809 02:11:25.074610 17179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:11:25.074802 17179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:11:25 dist-test-slave-p3t5 krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241485, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
Aug 09 02:11:25 dist-test-slave-p3t5 krb5kdc[16453](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241485, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:11:25.093942 17179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "66c5ba9d60754d68aa9197f15486cc1a"
format_stamp: "Formatted at 2026-08-09 02:11:25 on dist-test-slave-p3t5"
server_key: "44d9e18e3249ade6503a265377a1de43"
server_key_iv: "a0868217d55f8af3245fa86f1d750eef"
server_key_version: "kuduclusterkey@0"
I20260809 02:11:25.094467 17179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "66c5ba9d60754d68aa9197f15486cc1a"
format_stamp: "Formatted at 2026-08-09 02:11:25 on dist-test-slave-p3t5"
server_key: "44d9e18e3249ade6503a265377a1de43"
server_key_iv: "a0868217d55f8af3245fa86f1d750eef"
server_key_version: "kuduclusterkey@0"
I20260809 02:11:25.096729 17179 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:11:25.115125 17196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:11:25.117861 17179 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.001s
I20260809 02:11:25.118036 17179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "66c5ba9d60754d68aa9197f15486cc1a"
format_stamp: "Formatted at 2026-08-09 02:11:25 on dist-test-slave-p3t5"
server_key: "44d9e18e3249ade6503a265377a1de43"
server_key_iv: "a0868217d55f8af3245fa86f1d750eef"
server_key_version: "kuduclusterkey@0"
I20260809 02:11:25.118178 17179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:11:25.131302 17179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:11:25.136056 17179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:11:25.136243 17179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:11:25.136658 17179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:11:25.137099 17179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:11:25.137148 17179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:11:25.137180 17179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:11:25.137246 17179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:11:25.149873 17179 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:43299
I20260809 02:11:25.150420 17179 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:11:25.151399 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 17179
I20260809 02:11:25.151508 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:11:25.151686 5158 external_mini_cluster.cc:1428] Setting key 6ef3cba4186387cc7a100c795d8bf469
Aug 09 02:11:25 dist-test-slave-p3t5 krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241485, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:11:25.163837 17317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.151278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:37151 (local address 127.5.9.130:43361)
0809 02:11:25.151512 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 02:11:25.151561 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:11:25.152358 (+ 797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:11:25.152741 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:11:25.152746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:11:25.153267 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:11:25.153717 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:11:25.153725 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.154766 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.154767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:11:25.156200 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:11:25.156203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.156809 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.156811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:11:25.156963 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:25.157714 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:11:25.157726 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:11:25.159549 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:11:25.161756 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:11:25.161767 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:11:25.161770 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:11:25.162125 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:11:25.162362 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:11:25.162363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:11:25.162363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:11:25.162537 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:11:25.162858 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:11:25.162865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:11:25.163080 (+ 215us) client_negotiation.cc:769] Sending connection context
0809 02:11:25.163253 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 02:11:25.163552 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":93,"threads_started":1}
I20260809 02:11:25.164479 17175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.151283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:43361 (local address 127.5.9.190:37151)
0809 02:11:25.151341 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 02:11:25.151343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:11:25.151681 (+ 338us) server_negotiation.cc:407] Connection header received
0809 02:11:25.152499 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:11:25.152501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:11:25.152513 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:11:25.152544 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:11:25.153840 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.154636 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.156325 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.156627 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.156707 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:25.159672 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:11:25.159674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:11:25.159675 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:11:25.159684 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:11:25.161640 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:11:25.162219 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:11:25.162221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:11:25.162221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:11:25.162293 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:11:25.162611 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:11:25.162612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:11:25.162612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:11:25.162766 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:11:25.162829 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 02:11:25.164241 (+ 1412us) server_negotiation.cc:299] Negotiation successful
0809 02:11:25.164362 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 02:11:25.165436 17315 heartbeater.cc:347] Connected to a master server at 127.5.9.190:37151
I20260809 02:11:25.165603 17315 heartbeater.cc:464] Registering TS with master...
I20260809 02:11:25.165901 17315 heartbeater.cc:511] Master 127.5.9.190:37151 requested a full tablet report, sending...
I20260809 02:11:25.165921 17314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:43299 every 8 connection(s)
I20260809 02:11:25.166481 16946 ts_manager.cc:194] Registered new tserver with Master: 66c5ba9d60754d68aa9197f15486cc1a (127.5.9.130:43299)
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
I20260809 02:11:25.166998 16946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:43361
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:11:25.193375 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.9.148:49005/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:37151
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:11:25.344396 17321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:11:25.344811 17321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:11:25.344906 17321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:11:25.346999 17321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:11:25.347137 17321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:11:25.347229 17321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:11:25.349823 17321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.9.148:49005/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:37151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:11:25.350379 17321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:11:25.350756 17321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:11:25.358807 17327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:11:25.359680 17326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:11:25.360069 17321 server_base.cc:1034] running on GCE node
W20260809 02:11:25.360782 17329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:11:25.361117 17321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:11:25.361449 17321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:11:25.362643 17321 hybrid_clock.cc:648] HybridClock initialized: now 1786241485362620 us; error 37 us; skew 500 ppm
Aug 09 02:11:25 dist-test-slave-p3t5 krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241485, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:11:25.367148 17321 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:11:25.367784 17321 webserver.cc:460] Webserver started at http://127.5.9.131:38705/ using document root <none> and password file <none>
I20260809 02:11:25.368139 17321 fs_manager.cc:359] Metadata directory not provided
I20260809 02:11:25.368260 17321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:11:25.368466 17321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:11:25 dist-test-slave-p3t5 krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241485, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
I20260809 02:11:25.383577 17321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "a3ee0e018dd747bc9391db12e4573de0"
format_stamp: "Formatted at 2026-08-09 02:11:25 on dist-test-slave-p3t5"
server_key: "174e7dc809d1476edadbdcd174404cac"
server_key_iv: "c34be3f240e488b4d1eeb534d312fafa"
server_key_version: "kuduclusterkey@0"
I20260809 02:11:25.385283 17321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "a3ee0e018dd747bc9391db12e4573de0"
format_stamp: "Formatted at 2026-08-09 02:11:25 on dist-test-slave-p3t5"
server_key: "174e7dc809d1476edadbdcd174404cac"
server_key_iv: "c34be3f240e488b4d1eeb534d312fafa"
server_key_version: "kuduclusterkey@0"
I20260809 02:11:25.396389 17321 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.000s sys 0.002s
I20260809 02:11:25.420433 17336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:11:25.420789 17321 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260809 02:11:25.420921 17321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "a3ee0e018dd747bc9391db12e4573de0"
format_stamp: "Formatted at 2026-08-09 02:11:25 on dist-test-slave-p3t5"
server_key: "174e7dc809d1476edadbdcd174404cac"
server_key_iv: "c34be3f240e488b4d1eeb534d312fafa"
server_key_version: "kuduclusterkey@0"
I20260809 02:11:25.421044 17321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:11:25.457525 17321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:11:25.460266 17321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:11:25.460429 17321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:11:25.460836 17321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:11:25.461261 17321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:11:25.461339 17321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:11:25.461385 17321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:11:25.461402 17321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:11:25.472620 17321 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:35539
I20260809 02:11:25.473191 17321 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:11:25.482591 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 17321
I20260809 02:11:25.482707 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:11:25.482892 5158 external_mini_cluster.cc:1428] Setting key 3d6457e223fb6d44f0f1f6fb5e6a6686
I20260809 02:11:25.482012 17454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:35539 every 8 connection(s)
Aug 09 02:11:25 dist-test-slave-p3t5 krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241485, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:11:25.503508 17175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.474440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:55523 (local address 127.5.9.190:37151)
0809 02:11:25.474522 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 02:11:25.474525 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:11:25.485151 (+ 10626us) server_negotiation.cc:407] Connection header received
0809 02:11:25.492292 (+ 7141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:11:25.492294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:11:25.492312 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:11:25.492360 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:11:25.493475 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.494342 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.496408 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.496719 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.496765 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:25.499829 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:11:25.499830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:11:25.499831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:11:25.499839 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:11:25.501709 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:11:25.502258 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:11:25.502259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:11:25.502260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:11:25.502330 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:11:25.502667 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:11:25.502668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:11:25.502669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:11:25.502824 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:11:25.502871 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 02:11:25.503300 (+ 429us) server_negotiation.cc:299] Negotiation successful
0809 02:11:25.503405 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 02:11:25.504390 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.474375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:37151 (local address 127.5.9.131:55523)
0809 02:11:25.485008 (+ 10633us) negotiation.cc:107] Waiting for socket to connect
0809 02:11:25.485061 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 02:11:25.489798 (+ 4737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:11:25.492620 (+ 2822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:11:25.492626 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:11:25.493016 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:11:25.493333 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:11:25.493340 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.494490 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.494491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:11:25.496278 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:11:25.496282 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.496870 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.496871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:11:25.497081 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:25.497850 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:11:25.497863 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:11:25.499700 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:11:25.501816 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:11:25.501824 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:11:25.501826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:11:25.502155 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:11:25.502425 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:11:25.502426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:11:25.502426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:11:25.502580 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:11:25.502926 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:11:25.502932 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:11:25.503133 (+ 201us) client_negotiation.cc:769] Sending connection context
0809 02:11:25.503917 (+ 784us) client_negotiation.cc:240] Negotiation successful
0809 02:11:25.504115 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10582,"thread_start_us":86,"threads_started":1}
I20260809 02:11:25.505126 17455 heartbeater.cc:347] Connected to a master server at 127.5.9.190:37151
I20260809 02:11:25.505229 17455 heartbeater.cc:464] Registering TS with master...
I20260809 02:11:25.505455 17455 heartbeater.cc:511] Master 127.5.9.190:37151 requested a full tablet report, sending...
I20260809 02:11:25.506201 16946 ts_manager.cc:194] Registered new tserver with Master: a3ee0e018dd747bc9391db12e4573de0 (127.5.9.131:35539)
I20260809 02:11:25.506755 16946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:55523
I20260809 02:11:25.507922 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:11:25.519142 17175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.509907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33208 (local address 127.5.9.190:37151)
0809 02:11:25.509951 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:11:25.509953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:11:25.510438 (+ 485us) server_negotiation.cc:407] Connection header received
0809 02:11:25.510544 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:11:25.510545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:11:25.510556 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:11:25.510587 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:11:25.511421 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.512199 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.513420 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.513745 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.513791 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:25.515661 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:11:25.515662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:11:25.515663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:11:25.515671 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:11:25.517700 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:11:25.518183 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:11:25.518184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:11:25.518185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:11:25.518253 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:11:25.518494 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:11:25.518494 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:11:25.518495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:11:25.518639 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:11:25.518718 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:11:25.518949 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 02:11:25.519040 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:11:25.521721 16946 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33208:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:11:25.524286 16946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:11:25.535395 17470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.528681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:43299 (local address 127.0.0.1:51396)
0809 02:11:25.529869 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0809 02:11:25.529877 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 02:11:25.529919 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:11:25.530237 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:11:25.530241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:11:25.530244 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:11:25.530318 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:11:25.530319 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.532038 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.532040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:11:25.533871 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:11:25.533875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.535045 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.535046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:11:25.535134 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:25.535150 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 02:11:25.535226 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:11:25.535286 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1160,"thread_start_us":85,"threads_started":1}
I20260809 02:11:25.535910 17473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.528861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51396 (local address 127.5.9.130:43299)
0809 02:11:25.529664 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0809 02:11:25.529670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:11:25.529974 (+ 304us) server_negotiation.cc:407] Connection header received
0809 02:11:25.530032 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:11:25.530034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:11:25.530051 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:11:25.530125 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:11:25.530434 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.531933 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.534005 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.534834 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.534913 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:25.535005 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 02:11:25.535773 (+ 768us) server_negotiation.cc:299] Negotiation successful
0809 02:11:25.535822 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":730,"thread_start_us":96,"threads_started":1}
I20260809 02:11:25.537461 17248 tablet_service.cc:1467] Processing CreateTablet for tablet 2b247a38e5d54cd881282c127fb3ac78 (DEFAULT_TABLE table=test-table [id=6a14ee8d53a94fbd9a9fabca3204e00f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:11:25.537961 17248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b247a38e5d54cd881282c127fb3ac78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:11:25.541980 17472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.528204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:35539 (local address 127.0.0.1:50118)
0809 02:11:25.529363 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0809 02:11:25.529408 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:11:25.529478 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:11:25.537325 (+ 7847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:11:25.537326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:11:25.537329 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:11:25.537391 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:11:25.537393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.539120 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.539121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:11:25.540636 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:11:25.540640 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.541728 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.541729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:11:25.541799 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:25.541804 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:11:25.541839 (+ 35us) client_negotiation.cc:240] Negotiation successful
0809 02:11:25.541885 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1115,"thread_start_us":81,"threads_started":1}
I20260809 02:11:25.545815 17474 tablet_bootstrap.cc:492] T 2b247a38e5d54cd881282c127fb3ac78 P 66c5ba9d60754d68aa9197f15486cc1a: Bootstrap starting.
I20260809 02:11:25.546944 17468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.528485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:36619 (local address 127.0.0.1:58122)
0809 02:11:25.529721 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0809 02:11:25.529731 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:11:25.529780 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:11:25.542547 (+ 12767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:11:25.542548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:11:25.542551 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:11:25.542593 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:11:25.542595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.544207 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.544208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:11:25.545342 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:11:25.545346 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.546665 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.546666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:11:25.546760 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:25.546765 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:11:25.546802 (+ 37us) client_negotiation.cc:240] Negotiation successful
0809 02:11:25.546849 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1219,"thread_start_us":87,"threads_started":1}
I20260809 02:11:25.547652 17471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.528589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58122 (local address 127.5.9.129:36619)
0809 02:11:25.536869 (+ 8280us) server_negotiation.cc:206] Beginning negotiation
0809 02:11:25.536874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:11:25.536889 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:11:25.542405 (+ 5516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:11:25.542409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:11:25.542436 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:11:25.542536 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:11:25.542686 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.543773 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.545473 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.546515 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.547419 (+ 904us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:25.547496 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:11:25.547528 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 02:11:25.547579 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8234,"thread_start_us":64,"threads_started":1}
I20260809 02:11:25.547703 17474 tablet_bootstrap.cc:654] T 2b247a38e5d54cd881282c127fb3ac78 P 66c5ba9d60754d68aa9197f15486cc1a: Neither blocks nor log segments found. Creating new log.
I20260809 02:11:25.548324 17474 log.cc:824] T 2b247a38e5d54cd881282c127fb3ac78 P 66c5ba9d60754d68aa9197f15486cc1a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:11:25.548658 17102 tablet_service.cc:1467] Processing CreateTablet for tablet 2b247a38e5d54cd881282c127fb3ac78 (DEFAULT_TABLE table=test-table [id=6a14ee8d53a94fbd9a9fabca3204e00f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:11:25.549058 17102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b247a38e5d54cd881282c127fb3ac78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:11:25.554298 17469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.528320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50118 (local address 127.5.9.131:35539)
0809 02:11:25.537130 (+ 8810us) server_negotiation.cc:206] Beginning negotiation
0809 02:11:25.537134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:11:25.537146 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:11:25.537188 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:11:25.537190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:11:25.537208 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:11:25.537485 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:11:25.537671 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.539003 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.540774 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.541595 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.542438 (+ 843us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:25.554016 (+ 11578us) server_negotiation.cc:1035] Waiting for connection context
0809 02:11:25.554057 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 02:11:25.554122 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8759,"thread_start_us":111,"threads_started":1}
I20260809 02:11:25.555362 17377 tablet_service.cc:1467] Processing CreateTablet for tablet 2b247a38e5d54cd881282c127fb3ac78 (DEFAULT_TABLE table=test-table [id=6a14ee8d53a94fbd9a9fabca3204e00f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:11:25.555778 17377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b247a38e5d54cd881282c127fb3ac78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:11:25.558542 17474 tablet_bootstrap.cc:492] T 2b247a38e5d54cd881282c127fb3ac78 P 66c5ba9d60754d68aa9197f15486cc1a: No bootstrap required, opened a new log
I20260809 02:11:25.558645 17474 ts_tablet_manager.cc:1397] T 2b247a38e5d54cd881282c127fb3ac78 P 66c5ba9d60754d68aa9197f15486cc1a: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20260809 02:11:25.561313 17476 tablet_bootstrap.cc:492] T 2b247a38e5d54cd881282c127fb3ac78 P 4da26eecacfb484b839ec0e4b558db2a: Bootstrap starting.
I20260809 02:11:25.561503 17474 raft_consensus.cc:348] T 2b247a38e5d54cd881282c127fb3ac78 P 66c5ba9d60754d68aa9197f15486cc1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c5ba9d60754d68aa9197f15486cc1a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43299 } } peers { permanent_uuid: "a3ee0e018dd747bc9391db12e4573de0" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "4da26eecacfb484b839ec0e4b558db2a" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36619 } }
I20260809 02:11:25.561767 17474 raft_consensus.cc:374] T 2b247a38e5d54cd881282c127fb3ac78 P 66c5ba9d60754d68aa9197f15486cc1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:11:25.561821 17474 raft_consensus.cc:729] T 2b247a38e5d54cd881282c127fb3ac78 P 66c5ba9d60754d68aa9197f15486cc1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c5ba9d60754d68aa9197f15486cc1a, State: Initialized, Role: FOLLOWER
I20260809 02:11:25.561973 17474 consensus_queue.cc:260] T 2b247a38e5d54cd881282c127fb3ac78 P 66c5ba9d60754d68aa9197f15486cc1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c5ba9d60754d68aa9197f15486cc1a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43299 } } peers { permanent_uuid: "a3ee0e018dd747bc9391db12e4573de0" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "4da26eecacfb484b839ec0e4b558db2a" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36619 } }
I20260809 02:11:25.562282 17474 ts_tablet_manager.cc:1428] T 2b247a38e5d54cd881282c127fb3ac78 P 66c5ba9d60754d68aa9197f15486cc1a: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 02:11:25.562434 17315 heartbeater.cc:503] Master 127.5.9.190:37151 was elected leader, sending a full tablet report...
I20260809 02:11:25.563158 17476 tablet_bootstrap.cc:654] T 2b247a38e5d54cd881282c127fb3ac78 P 4da26eecacfb484b839ec0e4b558db2a: Neither blocks nor log segments found. Creating new log.
I20260809 02:11:25.563684 17476 log.cc:824] T 2b247a38e5d54cd881282c127fb3ac78 P 4da26eecacfb484b839ec0e4b558db2a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:11:25.564252 17477 tablet_bootstrap.cc:492] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0: Bootstrap starting.
I20260809 02:11:25.565847 17477 tablet_bootstrap.cc:654] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0: Neither blocks nor log segments found. Creating new log.
I20260809 02:11:25.566524 17477 log.cc:824] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0: Log is configured to *not* fsync() on all Append() calls
I20260809 02:11:25.567800 17476 tablet_bootstrap.cc:492] T 2b247a38e5d54cd881282c127fb3ac78 P 4da26eecacfb484b839ec0e4b558db2a: No bootstrap required, opened a new log
I20260809 02:11:25.568320 17476 ts_tablet_manager.cc:1397] T 2b247a38e5d54cd881282c127fb3ac78 P 4da26eecacfb484b839ec0e4b558db2a: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260809 02:11:25.571584 17476 raft_consensus.cc:348] T 2b247a38e5d54cd881282c127fb3ac78 P 4da26eecacfb484b839ec0e4b558db2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c5ba9d60754d68aa9197f15486cc1a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43299 } } peers { permanent_uuid: "a3ee0e018dd747bc9391db12e4573de0" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "4da26eecacfb484b839ec0e4b558db2a" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36619 } }
I20260809 02:11:25.571883 17476 raft_consensus.cc:374] T 2b247a38e5d54cd881282c127fb3ac78 P 4da26eecacfb484b839ec0e4b558db2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:11:25.571969 17476 raft_consensus.cc:729] T 2b247a38e5d54cd881282c127fb3ac78 P 4da26eecacfb484b839ec0e4b558db2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4da26eecacfb484b839ec0e4b558db2a, State: Initialized, Role: FOLLOWER
I20260809 02:11:25.572181 17476 consensus_queue.cc:260] T 2b247a38e5d54cd881282c127fb3ac78 P 4da26eecacfb484b839ec0e4b558db2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c5ba9d60754d68aa9197f15486cc1a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43299 } } peers { permanent_uuid: "a3ee0e018dd747bc9391db12e4573de0" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "4da26eecacfb484b839ec0e4b558db2a" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36619 } }
I20260809 02:11:25.572574 17476 ts_tablet_manager.cc:1428] T 2b247a38e5d54cd881282c127fb3ac78 P 4da26eecacfb484b839ec0e4b558db2a: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 02:11:25.572785 17173 heartbeater.cc:503] Master 127.5.9.190:37151 was elected leader, sending a full tablet report...
I20260809 02:11:25.573726 17477 tablet_bootstrap.cc:492] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0: No bootstrap required, opened a new log
I20260809 02:11:25.573807 17477 ts_tablet_manager.cc:1397] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.002s
I20260809 02:11:25.576012 17477 raft_consensus.cc:348] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c5ba9d60754d68aa9197f15486cc1a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43299 } } peers { permanent_uuid: "a3ee0e018dd747bc9391db12e4573de0" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "4da26eecacfb484b839ec0e4b558db2a" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36619 } }
I20260809 02:11:25.576200 17477 raft_consensus.cc:374] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:11:25.576232 17477 raft_consensus.cc:729] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3ee0e018dd747bc9391db12e4573de0, State: Initialized, Role: FOLLOWER
I20260809 02:11:25.576524 17477 consensus_queue.cc:260] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c5ba9d60754d68aa9197f15486cc1a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43299 } } peers { permanent_uuid: "a3ee0e018dd747bc9391db12e4573de0" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "4da26eecacfb484b839ec0e4b558db2a" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36619 } }
I20260809 02:11:25.576911 17477 ts_tablet_manager.cc:1428] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 02:11:25.577085 17455 heartbeater.cc:503] Master 127.5.9.190:37151 was elected leader, sending a full tablet report...
W20260809 02:11:25.622041 17174 tablet.cc:2369] T 2b247a38e5d54cd881282c127fb3ac78 P 4da26eecacfb484b839ec0e4b558db2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:11:25.664707 17482 raft_consensus.cc:482] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:11:25.664870 17482 raft_consensus.cc:504] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c5ba9d60754d68aa9197f15486cc1a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43299 } } peers { permanent_uuid: "a3ee0e018dd747bc9391db12e4573de0" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "4da26eecacfb484b839ec0e4b558db2a" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36619 } }
I20260809 02:11:25.665307 17482 leader_election.cc:288] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66c5ba9d60754d68aa9197f15486cc1a (127.5.9.130:43299), 4da26eecacfb484b839ec0e4b558db2a (127.5.9.129:36619)
W20260809 02:11:25.666554 17316 tablet.cc:2369] T 2b247a38e5d54cd881282c127fb3ac78 P 66c5ba9d60754d68aa9197f15486cc1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:11:25.670444 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.665685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:43299 (local address 127.5.9.131:41007)
0809 02:11:25.666156 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 02:11:25.666170 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:11:25.666224 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:11:25.666721 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:11:25.666741 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:11:25.666742 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:11:25.666833 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:11:25.666836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.668055 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.668056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:11:25.669302 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:11:25.669305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.670174 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.670175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:11:25.670258 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:25.670262 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:11:25.670309 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 02:11:25.670355 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449}
I20260809 02:11:25.671566 17473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.666379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:41007 (local address 127.5.9.130:43299)
0809 02:11:25.666500 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 02:11:25.666503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:11:25.666512 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:11:25.666544 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:11:25.666544 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:11:25.666556 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:11:25.666618 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:11:25.667198 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.667945 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.669412 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.670055 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.670128 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:25.670140 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:11:25.671389 (+ 1249us) server_negotiation.cc:299] Negotiation successful
0809 02:11:25.671446 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260809 02:11:25.671952 17269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b247a38e5d54cd881282c127fb3ac78" candidate_uuid: "a3ee0e018dd747bc9391db12e4573de0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66c5ba9d60754d68aa9197f15486cc1a" is_pre_election: true
I20260809 02:11:25.672144 17269 raft_consensus.cc:2454] T 2b247a38e5d54cd881282c127fb3ac78 P 66c5ba9d60754d68aa9197f15486cc1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3ee0e018dd747bc9391db12e4573de0 in term 0.
I20260809 02:11:25.673174 17338 leader_election.cc:302] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66c5ba9d60754d68aa9197f15486cc1a, a3ee0e018dd747bc9391db12e4573de0; no voters:
I20260809 02:11:25.673461 17482 raft_consensus.cc:2781] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:11:25.673524 17482 raft_consensus.cc:482] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:11:25.673548 17482 raft_consensus.cc:3037] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:11:25.675210 17482 raft_consensus.cc:504] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c5ba9d60754d68aa9197f15486cc1a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43299 } } peers { permanent_uuid: "a3ee0e018dd747bc9391db12e4573de0" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "4da26eecacfb484b839ec0e4b558db2a" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36619 } }
I20260809 02:11:25.675773 17269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b247a38e5d54cd881282c127fb3ac78" candidate_uuid: "a3ee0e018dd747bc9391db12e4573de0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66c5ba9d60754d68aa9197f15486cc1a"
I20260809 02:11:25.675905 17269 raft_consensus.cc:3037] T 2b247a38e5d54cd881282c127fb3ac78 P 66c5ba9d60754d68aa9197f15486cc1a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:11:25.677095 17269 raft_consensus.cc:2454] T 2b247a38e5d54cd881282c127fb3ac78 P 66c5ba9d60754d68aa9197f15486cc1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3ee0e018dd747bc9391db12e4573de0 in term 1.
I20260809 02:11:25.677810 17338 leader_election.cc:302] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66c5ba9d60754d68aa9197f15486cc1a, a3ee0e018dd747bc9391db12e4573de0; no voters:
I20260809 02:11:25.679950 17482 leader_election.cc:288] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [CANDIDATE]: Term 1 election: Requested vote from peers 66c5ba9d60754d68aa9197f15486cc1a (127.5.9.130:43299), 4da26eecacfb484b839ec0e4b558db2a (127.5.9.129:36619)
I20260809 02:11:25.680032 17482 raft_consensus.cc:2781] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:11:25.680066 17482 raft_consensus.cc:686] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [term 1 LEADER]: Becoming Leader. State: Replica: a3ee0e018dd747bc9391db12e4573de0, State: Running, Role: LEADER
I20260809 02:11:25.680160 17482 consensus_queue.cc:237] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c5ba9d60754d68aa9197f15486cc1a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43299 } } peers { permanent_uuid: "a3ee0e018dd747bc9391db12e4573de0" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "4da26eecacfb484b839ec0e4b558db2a" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36619 } }
I20260809 02:11:25.681070 16946 catalog_manager.cc:5495] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3ee0e018dd747bc9391db12e4573de0 (127.5.9.131). New cstate: current_term: 1 leader_uuid: "a3ee0e018dd747bc9391db12e4573de0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c5ba9d60754d68aa9197f15486cc1a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3ee0e018dd747bc9391db12e4573de0" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4da26eecacfb484b839ec0e4b558db2a" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36619 } health_report { overall_health: UNKNOWN } } }
I20260809 02:11:25.682590 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.665800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:36619 (local address 127.5.9.131:39437)
0809 02:11:25.670560 (+ 4760us) negotiation.cc:107] Waiting for socket to connect
0809 02:11:25.670574 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:11:25.670641 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:11:25.672040 (+ 1399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:11:25.672042 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:11:25.672044 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:11:25.672081 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:11:25.672082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.678098 (+ 6016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.678100 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:11:25.679162 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:11:25.679166 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.682326 (+ 3160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.682327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:11:25.682408 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:25.682412 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:11:25.682462 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 02:11:25.682512 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4724,"thread_start_us":129,"threads_started":1}
I20260809 02:11:25.682950 17471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.666433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:39437 (local address 127.5.9.129:36619)
0809 02:11:25.666681 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 02:11:25.666683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:11:25.670880 (+ 4197us) server_negotiation.cc:407] Connection header received
0809 02:11:25.670933 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:11:25.670935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:11:25.670947 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:11:25.671014 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:11:25.672173 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.672976 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.679286 (+ 6310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.679843 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.679922 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:25.679935 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 02:11:25.682806 (+ 2871us) server_negotiation.cc:299] Negotiation successful
0809 02:11:25.682873 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223}
I20260809 02:11:25.683657 17123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b247a38e5d54cd881282c127fb3ac78" candidate_uuid: "a3ee0e018dd747bc9391db12e4573de0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4da26eecacfb484b839ec0e4b558db2a" is_pre_election: true
I20260809 02:11:25.683925 17123 raft_consensus.cc:2454] T 2b247a38e5d54cd881282c127fb3ac78 P 4da26eecacfb484b839ec0e4b558db2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3ee0e018dd747bc9391db12e4573de0 in term 0.
I20260809 02:11:25.684109 17122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b247a38e5d54cd881282c127fb3ac78" candidate_uuid: "a3ee0e018dd747bc9391db12e4573de0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4da26eecacfb484b839ec0e4b558db2a"
I20260809 02:11:25.684175 17122 raft_consensus.cc:3037] T 2b247a38e5d54cd881282c127fb3ac78 P 4da26eecacfb484b839ec0e4b558db2a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:11:25.685416 17122 raft_consensus.cc:2454] T 2b247a38e5d54cd881282c127fb3ac78 P 4da26eecacfb484b839ec0e4b558db2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3ee0e018dd747bc9391db12e4573de0 in term 1.
I20260809 02:11:25.719548 17469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.715878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50124 (local address 127.5.9.131:35539)
0809 02:11:25.715923 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 02:11:25.715925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:11:25.715933 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:11:25.715976 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:11:25.715977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:11:25.716001 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:11:25.716076 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:11:25.716324 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.717079 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.718221 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:11:25.718542 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:11:25.718618 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:11:25.718884 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:11:25.719035 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:11:25.719119 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 02:11:25.719406 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 02:11:25.719434 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 02:11:25 dist-test-slave-p3t5 krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241485, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
I20260809 02:11:25.722008 17122 raft_consensus.cc:1261] T 2b247a38e5d54cd881282c127fb3ac78 P 4da26eecacfb484b839ec0e4b558db2a [term 1 FOLLOWER]: Refusing update from remote peer a3ee0e018dd747bc9391db12e4573de0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:11:25.722636 17484 consensus_queue.cc:1035] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4da26eecacfb484b839ec0e4b558db2a" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 36619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:11:25.725334 17269 raft_consensus.cc:1261] T 2b247a38e5d54cd881282c127fb3ac78 P 66c5ba9d60754d68aa9197f15486cc1a [term 1 FOLLOWER]: Refusing update from remote peer a3ee0e018dd747bc9391db12e4573de0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:11:25.725780 17484 consensus_queue.cc:1035] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66c5ba9d60754d68aa9197f15486cc1a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:11:25.730124 17487 mvcc.cc:205] Tried to move back new op lower bound from 7316445125513834496 to 7316445125346541568. Current Snapshot: MvccSnapshot[applied={T|T < 7316445125513834496}]
I20260809 02:11:25.730896 17491 mvcc.cc:205] Tried to move back new op lower bound from 7316445125513834496 to 7316445125346541568. Current Snapshot: MvccSnapshot[applied={T|T < 7316445125513834496}]
I20260809 02:11:25.735340 16946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:11:25.735486 16946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:11:25.736507 16946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 945412525acb4fdabd8219e527c5cc5a: Sending DeleteTablet for 3 replicas of tablet 2b247a38e5d54cd881282c127fb3ac78
I20260809 02:11:25.737025 17377 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b247a38e5d54cd881282c127fb3ac78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:11:25 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:50118
I20260809 02:11:25.737355 17102 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b247a38e5d54cd881282c127fb3ac78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:11:25 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:58122
I20260809 02:11:25.737900 17248 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b247a38e5d54cd881282c127fb3ac78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:11:25 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:51396
I20260809 02:11:25.738721 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 17033
W20260809 02:11:25.748494 17339 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:36619 (error 108)
W20260809 02:11:25.749818 16932 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:36619 (error 108)
I20260809 02:11:25.750885 17468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.750750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:36619 (local address 127.0.0.1:58136)
0809 02:11:25.750808 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 02:11:25.750845 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:36619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 02:11:25.751138 16932 catalog_manager.cc:4584] TS 4da26eecacfb484b839ec0e4b558db2a (127.5.9.129:36619): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b247a38e5d54cd881282c127fb3ac78: Network error: Client connection negotiation failed: client connection to 127.5.9.129:36619: connect: Connection refused (error 111)
I20260809 02:11:25.751446 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 17179
I20260809 02:11:25.760565 17498 tablet_replica.cc:330] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0: stopping tablet replica
I20260809 02:11:25.760833 17498 raft_consensus.cc:2229] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:11:25.761065 17498 raft_consensus.cc:2258] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:11:25.761783 17498 ts_tablet_manager.cc:1905] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:11:25.762324 17483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:11:25.762151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:36619 (local address 127.5.9.131:43011)
0809 02:11:25.762219 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 02:11:25.762271 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:36619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 02:11:25.764788 17498 ts_tablet_manager.cc:1918] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:11:25.764950 17498 log.cc:1196] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/2b247a38e5d54cd881282c127fb3ac78
I20260809 02:11:25.765410 17498 ts_tablet_manager.cc:1939] T 2b247a38e5d54cd881282c127fb3ac78 P a3ee0e018dd747bc9391db12e4573de0: Deleting consensus metadata
I20260809 02:11:25.766011 16930 catalog_manager.cc:4841] TS a3ee0e018dd747bc9391db12e4573de0 (127.5.9.131:35539): tablet 2b247a38e5d54cd881282c127fb3ac78 (table test-table [id=6a14ee8d53a94fbd9a9fabca3204e00f]) successfully deleted
W20260809 02:11:25.767001 17468 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:11:25.766585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:43299 (local address 127.0.0.1:51410)
0809 02:11:25.766638 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 02:11:25.766650 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:11:25.766690 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:11:25.766942 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.130:43299: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 02:11:25.767684 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 17321
I20260809 02:11:25.778779 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 16914
I20260809 02:11:25.803014 5158 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 02:11:26.158613 5158 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 02:11:26.158838 5158 mini_ranger.cc:66] Stopping Ranger...
I20260809 02:11:26.504556 5158 mini_ranger.cc:68] Stopped Ranger
2026-08-09 02:11:26.504 UTC [16483] LOG: received smart shutdown request
2026-08-09 02:11:26.509 UTC [16483] LOG: background worker "logical replication launcher" (PID 16492) exited with exit code 1
2026-08-09 02:11:26.509 UTC [16487] LOG: shutting down
2026-08-09 02:11:26.699 UTC [16483] LOG: database system is shut down
2026-08-09T02:11:26Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50523 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:11:26 dist-test-slave-p3t5 krb5kdc[17511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:11:26 dist-test-slave-p3t5 krb5kdc[17511](info): set up 2 sockets
Aug 09 02:11:26 dist-test-slave-p3t5 krb5kdc[17511](info): commencing operation
krb5kdc: starting...
W20260809 02:11:27.905907 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:11:27 dist-test-slave-p3t5 krb5kdc[17511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:11:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:11:28Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.9.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.9.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.9.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/rangeradmin_127.5.9.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.9.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/rangeradmin_127.5.9.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.9.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.9.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.9.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/rangerlookup_127.5.9.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.9.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/rangerlookup_127.5.9.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.9.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.9.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.9.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/HTTP_127.5.9.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.9.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/HTTP_127.5.9.148@KRBTEST.COM.keytab.
I20260809 02:11:28.097460 5158 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasktks196/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/postgres -l logfile start
2026-08-09 02:11:30.774 UTC [17541] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 02:11:30.774 UTC [17541] LOG: listening on IPv4 address "127.5.9.148", port 53467
2026-08-09 02:11:30.778 UTC [17541] LOG: listening on Unix socket "/tmp/.s.PGSQL.53467"
2026-08-09 02:11:30.803 UTC [17544] LOG: database system was shut down at 2026-08-09 02:11:29 UTC
2026-08-09 02:11:30.808 UTC [17541] LOG: database system is ready to accept connections
I20260809 02:11:31.786262 5158 mini_postgres.cc:96] Postgres bound to 53467
2026-08-09 02:11:31.794 UTC [17552] FATAL: database "slave" does not exist
127.5.9.148:53467 - accepting connections
I20260809 02:11:31.794940 5158 mini_ranger.cc:161] Starting Ranger...
I20260809 02:11:31.811884 5158 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 02:11:32.366735 5158 mini_ranger.cc:87] Created ranger Postgres database
I20260809 02:11:32.366873 5158 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-admin
2026-08-09 02:11:32,696 [I] DB FLAVOR :POSTGRES
2026-08-09 02:11:32,697 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 02:11:32,697 [I] Checking connection..
2026-08-09 02:11:32,697 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 02:11:32,973 [I] Checking connection passed.
2026-08-09 02:11:32,973 [I] --------- Verifying version history table ---------
2026-08-09 02:11:32,974 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:11:33,249 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 02:11:33,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:11:33,519 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 02:11:33,519 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 02:11:33,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 02:11:33.771 UTC [17618] WARNING: there is no transaction in progress
2026-08-09 02:11:33,779 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 02:11:33,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:11:34,042 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 02:11:34,043 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 02:11:34,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 02:11:34,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 02:11:34,580 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-p3t5.c.gcp-upstream.internal','N') ;"
2026-08-09 02:11:34,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 02:11:35,125 [I] Table x_portal_user does not exist in database ranger
2026-08-09 02:11:35,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 02:11:35,378 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 02:11:35,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 02:11:35,644 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 02:11:35,920 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 02:11:35,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 02:11:37.014 UTC [17733] WARNING: there is no transaction in progress
2026-08-09 02:11:37.032 UTC [17733] WARNING: there is no transaction in progress
2026-08-09 02:11:37.051 UTC [17733] WARNING: there is no transaction in progress
2026-08-09 02:11:37.069 UTC [17733] WARNING: there is no transaction in progress
2026-08-09 02:11:37.087 UTC [17733] WARNING: there is no transaction in progress
2026-08-09 02:11:37.107 UTC [17733] WARNING: there is no transaction in progress
2026-08-09 02:11:37.171 UTC [17733] WARNING: there is no transaction in progress
2026-08-09 02:11:37.178 UTC [17733] WARNING: there is no transaction in progress
2026-08-09 02:11:37.193 UTC [17733] WARNING: there is no transaction in progress
2026-08-09 02:11:37.204 UTC [17733] WARNING: there is no transaction in progress
2026-08-09 02:11:37.215 UTC [17733] WARNING: there is no transaction in progress
2026-08-09 02:11:37.230 UTC [17733] WARNING: there is no transaction in progress
2026-08-09 02:11:37.240 UTC [17733] WARNING: there is no transaction in progress
2026-08-09 02:11:37.247 UTC [17733] WARNING: there is no transaction in progress
2026-08-09 02:11:37.254 UTC [17733] WARNING: there is no transaction in progress
2026-08-09 02:11:37.262 UTC [17733] WARNING: there is no transaction in progress
2026-08-09 02:11:37,999 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 02:11:38,000 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 02:11:38,242 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 02:11:38,242 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 02:11:38,493 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 02:11:38,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 02:11:38,775 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 02:11:38,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 02:11:39,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 02:11:39,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-p3t5.c.gcp-upstream.internal';"
2026-08-09 02:11:39,564 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 02:11:39,564 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 02:11:39,824 [I] DB_PATCHES have already been applied
I20260809 02:11:39.830984 5158 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-admin:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasktks196/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 02:11:39.831071 5158 mini_ranger.cc:193] Using host: 127.5.9.148
I20260809 02:11:39.835405 5158 mini_ranger.cc:227] Ranger admin URL: http://127.5.9.148:48901
Aug 09, 2026 2:11:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 2:11:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 2:11:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 2:11:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 2:11:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 2:11:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 2:11:40 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 02:11:40.979065 5158 mini_ranger.cc:160] Time spent starting Ranger: real 9.184s user 0.000s sys 0.012s
Aug 09, 2026 2:12:01 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 2:12:01 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 09, 2026 2:12:01 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 2:12:01 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 2:12:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 2:12:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 2:12:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 2:12:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 2:12:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 2:12:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 2:12:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 2:12:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 2:12:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 2:12:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 2:12:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 2:12:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 2:12:01 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 2:12:02 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 02:12:03.135133 5158 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.9.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.9.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.9.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/rangerkms_127.5.9.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.9.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/rangerkms_127.5.9.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.9.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.9.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.9.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/HTTP_127.5.9.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.9.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/HTTP_127.5.9.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 02:12:03 dist-test-slave-p3t5 krb5kdc[17511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241523, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 02:12:03.236457 5158 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 02:12:03.248457 5158 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 02:12:04.223541 5158 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 02:12:04.223646 5158 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-kms
I20260809 02:12:04.223670 5158 mini_ranger_kms.cc:227] Using postgres at 127.5.9.148:53467
2026-08-09 02:12:04,421 [I] DB FLAVOR :POSTGRES
2026-08-09 02:12:04,421 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 02:12:04,421 [I] Checking connection
2026-08-09 02:12:04,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 02:12:04,688 [I] connection success
2026-08-09 02:12:04,688 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 02:12:04,688 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 02:12:04,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 02:12:04,941 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 02:12:04,942 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 02:12:04,942 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 02:12:04,942 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.9.148:53467/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 02:12:05,214 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 02:12:05.730320 5158 mini_ranger_kms.cc:310] Created kms service
I20260809 02:12:05.948890 5158 mini_ranger_kms.cc:326] Created kudu user
I20260809 02:12:06.001327 5158 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 02:12:05 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241523, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
I20260809 02:12:06.461957 5158 mini_ranger_kms.cc:384] Added ranger policy
I20260809 02:12:06.462080 5158 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-kms:/tmp/dist-test-tasktks196/build/release/bin/postgresql.jar:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasktks196/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 02:12:06.462105 5158 mini_ranger_kms.cc:240] Using host: 127.5.9.148
I20260809 02:12:06.467180 5158 mini_ranger_kms.cc:276] Ranger KMS PID: 17924
I20260809 02:12:06.467284 5158 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.9.148:46207
Aug 09, 2026 2:12:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 2:12:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 2:12:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 2:12:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 2:12:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 2:12:08 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 02:12:08.853387 5158 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.617s user 0.020s sys 0.161s
I20260809 02:12:08.853500 5158 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 02:12:08.853552 5158 mini_ranger_kms.cc:401] 127.5.9.148:46207/kms/v1/keys
Aug 09 02:12:09 dist-test-slave-p3t5 krb5kdc[17511](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241529, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.9.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 02:12:09 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241529, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.9.148@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
Aug 09 02:12:10 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241529, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.9.148@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
Aug 09 02:12:12 dist-test-slave-p3t5 krb5kdc[17511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:12:12.757957 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:33853
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:33853
--ranger_config_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.9.148:46207/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:12.850893 17978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:12.851169 17978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:12.851195 17978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:12.853449 17978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:12:12.853525 17978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:12.853540 17978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:12.853554 17978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:12:12.853569 17978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:12:12.855532 17978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.9.148:46207/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:33853
--ranger_config_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:33853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:12.855870 17978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:12.856150 17978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:12.861822 17984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:12.861842 17983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:12.861958 17978 server_base.cc:1034] running on GCE node
W20260809 02:12:12.861914 17986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:12.862392 17978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:12.862736 17978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:12.863925 17978 hybrid_clock.cc:648] HybridClock initialized: now 1786241532863894 us; error 51 us; skew 500 ppm
Aug 09 02:12:12 dist-test-slave-p3t5 krb5kdc[17511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241532, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:12.866840 17978 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:12:12.867362 17978 webserver.cc:460] Webserver started at http://127.5.9.190:45941/ using document root <none> and password file <none>
I20260809 02:12:12.867615 17978 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:12.867681 17978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:12.867857 17978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:12:12 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241532, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
I20260809 02:12:12.932637 17978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "284c0350e1b24f508c56744bc7cbff44"
format_stamp: "Formatted at 2026-08-09 02:12:12 on dist-test-slave-p3t5"
server_key: "065824d07b0a29b69959fffba84047c8"
server_key_iv: "cf166239ba59a58bdf5fd2ad89533b39"
server_key_version: "kuduclusterkey@0"
I20260809 02:12:12.933301 17978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "284c0350e1b24f508c56744bc7cbff44"
format_stamp: "Formatted at 2026-08-09 02:12:12 on dist-test-slave-p3t5"
server_key: "065824d07b0a29b69959fffba84047c8"
server_key_iv: "cf166239ba59a58bdf5fd2ad89533b39"
server_key_version: "kuduclusterkey@0"
I20260809 02:12:12.935999 17978 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.005s sys 0.000s
I20260809 02:12:12.955438 17993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:12.955737 17978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:12.955865 17978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "284c0350e1b24f508c56744bc7cbff44"
format_stamp: "Formatted at 2026-08-09 02:12:12 on dist-test-slave-p3t5"
server_key: "065824d07b0a29b69959fffba84047c8"
server_key_iv: "cf166239ba59a58bdf5fd2ad89533b39"
server_key_version: "kuduclusterkey@0"
I20260809 02:12:12.955986 17978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:12.983271 17978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:12.986236 17978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:12.986495 17978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:12.992726 17978 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:33853
I20260809 02:12:12.992785 18047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:33853 every 8 connection(s)
I20260809 02:12:12.993269 17978 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:12:12.994231 18049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:12.996591 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 17978
I20260809 02:12:12.996696 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:12:12.996927 5158 external_mini_cluster.cc:1428] Setting key 2c720efa5120039cb373d5d1826a6de2
I20260809 02:12:12.998327 18049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44: Bootstrap starting.
I20260809 02:12:12.999146 18049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:12.999518 18049 log.cc:824] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:13.000554 18049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44: No bootstrap required, opened a new log
I20260809 02:12:13.004529 18049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284c0350e1b24f508c56744bc7cbff44" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33853 } }
I20260809 02:12:13.004766 18049 raft_consensus.cc:374] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:13.004817 18049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284c0350e1b24f508c56744bc7cbff44, State: Initialized, Role: FOLLOWER
I20260809 02:12:13.004985 18049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284c0350e1b24f508c56744bc7cbff44" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33853 } }
I20260809 02:12:13.005100 18049 raft_consensus.cc:388] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:12:13.005144 18049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:12:13.005189 18049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:13.006206 18049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284c0350e1b24f508c56744bc7cbff44" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33853 } }
I20260809 02:12:13.006352 18049 leader_election.cc:302] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 284c0350e1b24f508c56744bc7cbff44; no voters:
I20260809 02:12:13.006634 18049 leader_election.cc:288] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:12:13.006714 18054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:12:13.006938 18054 raft_consensus.cc:686] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 1 LEADER]: Becoming Leader. State: Replica: 284c0350e1b24f508c56744bc7cbff44, State: Running, Role: LEADER
I20260809 02:12:13.007035 18049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:12:13.007095 18054 consensus_queue.cc:237] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284c0350e1b24f508c56744bc7cbff44" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33853 } }
I20260809 02:12:13.007526 18054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 284c0350e1b24f508c56744bc7cbff44. Latest consensus state: current_term: 1 leader_uuid: "284c0350e1b24f508c56744bc7cbff44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284c0350e1b24f508c56744bc7cbff44" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33853 } } }
I20260809 02:12:13.007536 18056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "284c0350e1b24f508c56744bc7cbff44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284c0350e1b24f508c56744bc7cbff44" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33853 } } }
I20260809 02:12:13.007716 18054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [sys.catalog]: This master's current role is: LEADER
I20260809 02:12:13.007728 18056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [sys.catalog]: This master's current role is: LEADER
Aug 09 02:12:13 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:13.009006 18058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:12:13.009758 18058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:12:13.010125 18049 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 02:12:13.020560 18058 catalog_manager.cc:1347] Generated new cluster ID: da3883d463bb449791de6f54ab13bab1
I20260809 02:12:13.020732 18058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:12:13.021183 18052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:12.997583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60564 (local address 127.5.9.190:33853)
0809 02:12:12.999884 (+ 2301us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:12.999895 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:12.999945 (+ 50us) server_negotiation.cc:407] Connection header received
0809 02:12:13.000567 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:13.000578 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:13.000653 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:13.000843 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:13.001708 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:13.003150 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:13.004352 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:13.005075 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:13.005204 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:13.008661 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:13.008667 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:13.008667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:13.008683 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:13.016658 (+ 7975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:13.017244 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:13.017248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:13.017248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:13.017451 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:13.017769 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:13.017770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:13.017771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:13.018690 (+ 919us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:13.019572 (+ 882us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:13.019678 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 02:12:13.020770 (+ 1092us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2244,"thread_start_us":110,"threads_started":1}
I20260809 02:12:13.031486 18058 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:12:13.032382 18058 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:12:13.040302 18058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44: Generated new TSK 0
I20260809 02:12:13.040553 18058 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:12:13.793357 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.9.148:46207/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:33853
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:13.914525 18095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:13.914837 18095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:13.914889 18095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:13.916361 18095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:13.916455 18095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:13.916513 18095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:12:13.918454 18095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.9.148:46207/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:33853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:13.918768 18095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:13.919065 18095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:13.925830 18104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:13.926170 18095 server_base.cc:1034] running on GCE node
W20260809 02:12:13.925832 18102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:13.926136 18101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:13.926602 18095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:13.926965 18095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:13.928172 18095 hybrid_clock.cc:648] HybridClock initialized: now 1786241533928149 us; error 41 us; skew 500 ppm
Aug 09 02:12:13 dist-test-slave-p3t5 krb5kdc[17511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241533, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:13.932525 18095 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:12:13.933178 18095 webserver.cc:460] Webserver started at http://127.5.9.129:33113/ using document root <none> and password file <none>
I20260809 02:12:13.933568 18095 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:13.933679 18095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:13.933903 18095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:12:13 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241533, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
I20260809 02:12:13.952760 18095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "3e6017ba692c44e4a710ee050b629392"
format_stamp: "Formatted at 2026-08-09 02:12:13 on dist-test-slave-p3t5"
server_key: "387ac71243d53e7d6fa32ed88e948b75"
server_key_iv: "568eff03226691ec904bf72166257e1d"
server_key_version: "kuduclusterkey@0"
I20260809 02:12:13.953274 18095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "3e6017ba692c44e4a710ee050b629392"
format_stamp: "Formatted at 2026-08-09 02:12:13 on dist-test-slave-p3t5"
server_key: "387ac71243d53e7d6fa32ed88e948b75"
server_key_iv: "568eff03226691ec904bf72166257e1d"
server_key_version: "kuduclusterkey@0"
I20260809 02:12:13.958017 18095 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:12:13.974798 18113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:13.975086 18095 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:13.975191 18095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "3e6017ba692c44e4a710ee050b629392"
format_stamp: "Formatted at 2026-08-09 02:12:13 on dist-test-slave-p3t5"
server_key: "387ac71243d53e7d6fa32ed88e948b75"
server_key_iv: "568eff03226691ec904bf72166257e1d"
server_key_version: "kuduclusterkey@0"
I20260809 02:12:13.975293 18095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:13.990486 18095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:13.994302 18095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:13.994495 18095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:13.994750 18095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:13.995148 18095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:12:13.995187 18095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:13.995213 18095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:12:13.995226 18095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:14.001874 18095 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:44433
I20260809 02:12:14.002005 18231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:44433 every 8 connection(s)
I20260809 02:12:14.002345 18095 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:12:14 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241533, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:14.012261 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 18095
I20260809 02:12:14.012377 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:12:14.012583 5158 external_mini_cluster.cc:1428] Setting key 1250ed3869ff1457458904f2a4bea15f
I20260809 02:12:14.015092 18234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:14.003207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:33853 (local address 127.5.9.129:56495)
0809 02:12:14.003546 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:14.003591 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:14.004265 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:14.004554 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:14.004559 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:14.004830 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:14.005135 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:14.005142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:14.005909 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:14.005911 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:14.007239 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:14.007242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:14.007783 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:14.007784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:14.007959 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:14.008654 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:14.008666 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:14.010603 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:14.012893 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:14.012904 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:14.012907 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:14.013267 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:14.013523 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:14.013524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:14.013524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:14.013829 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:14.014224 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:14.014233 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:14.014481 (+ 248us) client_negotiation.cc:769] Sending connection context
0809 02:12:14.014611 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 02:12:14.014812 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":92,"threads_started":1}
I20260809 02:12:14.015666 18235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:14.003272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:56495 (local address 127.5.9.190:33853)
0809 02:12:14.003477 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:14.003479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:14.003688 (+ 209us) server_negotiation.cc:407] Connection header received
0809 02:12:14.004377 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:14.004378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:14.004391 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:14.004469 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:14.005237 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:14.005807 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:14.007338 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:14.007667 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:14.007749 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:14.010729 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:14.010731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:14.010731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:14.010740 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:14.012772 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:14.013362 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:14.013364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:14.013364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:14.013450 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:14.013930 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:14.013931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:14.013932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:14.014116 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:14.014189 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:14.015269 (+ 1080us) server_negotiation.cc:299] Negotiation successful
0809 02:12:14.015460 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":79,"threads_started":1}
I20260809 02:12:14.017074 18232 heartbeater.cc:347] Connected to a master server at 127.5.9.190:33853
I20260809 02:12:14.017259 18232 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:14.017566 18232 heartbeater.cc:511] Master 127.5.9.190:33853 requested a full tablet report, sending...
I20260809 02:12:14.018231 18002 ts_manager.cc:194] Registered new tserver with Master: 3e6017ba692c44e4a710ee050b629392 (127.5.9.129:44433)
I20260809 02:12:14.019160 18002 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:56495
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:12:14.062249 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.9.148:46207/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:33853
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:12:14 dist-test-slave-p3t5 krb5kdc[17511](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241534, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 02:12:14.191695 18240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:14.192097 18240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:14.192152 18240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:14.193957 18240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:14.194109 18240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:14.194208 18240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:12:14.196662 18240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.9.148:46207/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:33853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:14.196952 18240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:14.197297 18240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:14.204038 18248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:14.204171 18250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:14.204051 18247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:14.205089 18240 server_base.cc:1034] running on GCE node
I20260809 02:12:14.205323 18240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:14.205662 18240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:14.206856 18240 hybrid_clock.cc:648] HybridClock initialized: now 1786241534206829 us; error 40 us; skew 500 ppm
Aug 09 02:12:14 dist-test-slave-p3t5 krb5kdc[17511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241534, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:14.210688 18240 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:12:14.211318 18240 webserver.cc:460] Webserver started at http://127.5.9.130:33931/ using document root <none> and password file <none>
I20260809 02:12:14.211625 18240 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:14.211694 18240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:14.211899 18240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:12:14 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241534, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
I20260809 02:12:14.226706 18240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "011d00facd21485d82152593a6c28345"
format_stamp: "Formatted at 2026-08-09 02:12:14 on dist-test-slave-p3t5"
server_key: "0b3ee305a6a9922146ca25680c795d4c"
server_key_iv: "e7d874bcd9752c44abba6345ad7d9970"
server_key_version: "kuduclusterkey@0"
I20260809 02:12:14.227352 18240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "011d00facd21485d82152593a6c28345"
format_stamp: "Formatted at 2026-08-09 02:12:14 on dist-test-slave-p3t5"
server_key: "0b3ee305a6a9922146ca25680c795d4c"
server_key_iv: "e7d874bcd9752c44abba6345ad7d9970"
server_key_version: "kuduclusterkey@0"
I20260809 02:12:14.229549 18240 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:12:14.252020 18257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:14.252389 18240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:12:14.252583 18240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "011d00facd21485d82152593a6c28345"
format_stamp: "Formatted at 2026-08-09 02:12:14 on dist-test-slave-p3t5"
server_key: "0b3ee305a6a9922146ca25680c795d4c"
server_key_iv: "e7d874bcd9752c44abba6345ad7d9970"
server_key_version: "kuduclusterkey@0"
I20260809 02:12:14.252794 18240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:14.276526 18240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:14.279359 18240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:14.279608 18240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:14.279960 18240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:14.280498 18240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:12:14.280607 18240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:14.280694 18240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:12:14.280754 18240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:14.288131 18240 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:42321
I20260809 02:12:14.289019 18240 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:12:14.291638 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 18240
I20260809 02:12:14.291759 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:12:14.292017 5158 external_mini_cluster.cc:1428] Setting key 2114c92f8c83b80b6ce00f4226537766
I20260809 02:12:14.300038 18375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:42321 every 8 connection(s)
Aug 09 02:12:14 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241534, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
I20260809 02:12:14.315403 18235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:14.301317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:59583 (local address 127.5.9.190:33853)
0809 02:12:14.301368 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:14.301370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:14.303468 (+ 2098us) server_negotiation.cc:407] Connection header received
0809 02:12:14.304451 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:14.304453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:14.304469 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:14.304503 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:14.305478 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:14.306248 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:14.307883 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:14.308197 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:14.308247 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:14.311505 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:14.311507 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:14.311508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:14.311518 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:14.313470 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:14.314116 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:14.314118 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:14.314118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:14.314189 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:14.314509 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:14.314510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:14.314510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:14.314682 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:14.314736 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:14.315183 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 02:12:14.315291 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:12:14.316277 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:14.300970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:33853 (local address 127.5.9.130:59583)
0809 02:12:14.303288 (+ 2318us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:14.303348 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:14.304149 (+ 801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:14.304715 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:14.304719 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:14.305035 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:14.305351 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:14.305359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:14.306379 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:14.306379 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:14.307767 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:14.307771 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:14.308537 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:14.308540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:14.308781 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:14.309488 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:14.309500 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:14.311371 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:14.313590 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:14.313601 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:14.313604 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:14.314015 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:14.314267 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:14.314268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:14.314268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:14.314423 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:14.314776 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:14.314782 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:14.315000 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 02:12:14.315800 (+ 800us) client_negotiation.cc:240] Negotiation successful
0809 02:12:14.316017 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2261,"thread_start_us":126,"threads_started":1}
I20260809 02:12:14.317022 18376 heartbeater.cc:347] Connected to a master server at 127.5.9.190:33853
I20260809 02:12:14.317128 18376 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:14.317340 18376 heartbeater.cc:511] Master 127.5.9.190:33853 requested a full tablet report, sending...
I20260809 02:12:14.317811 18002 ts_manager.cc:194] Registered new tserver with Master: 011d00facd21485d82152593a6c28345 (127.5.9.130:42321)
I20260809 02:12:14.318328 18002 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:59583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:12:14.362478 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.9.148:46207/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:33853
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:14.531752 18381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:14.532173 18381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:14.532269 18381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:14.534301 18381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:14.534456 18381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:14.534554 18381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:12:14.537132 18381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.9.148:46207/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:33853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:14.537693 18381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:14.538056 18381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:14.545876 18389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:14.546093 18386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:14.546430 18387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:14.546631 18381 server_base.cc:1034] running on GCE node
I20260809 02:12:14.546916 18381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:14.547243 18381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:14.548440 18381 hybrid_clock.cc:648] HybridClock initialized: now 1786241534548413 us; error 40 us; skew 500 ppm
Aug 09 02:12:14 dist-test-slave-p3t5 krb5kdc[17511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241534, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:14.552479 18381 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:12:14.553170 18381 webserver.cc:460] Webserver started at http://127.5.9.131:36795/ using document root <none> and password file <none>
I20260809 02:12:14.553534 18381 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:14.553704 18381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:14.553938 18381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:12:14 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241534, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
I20260809 02:12:14.573364 18381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "3721555f5e494cb39912b96f4e3622bb"
format_stamp: "Formatted at 2026-08-09 02:12:14 on dist-test-slave-p3t5"
server_key: "0e65b716bb9285986b8de94f38c6c09c"
server_key_iv: "e7eee5304c64648134675841bd2863b9"
server_key_version: "kuduclusterkey@0"
I20260809 02:12:14.574035 18381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "3721555f5e494cb39912b96f4e3622bb"
format_stamp: "Formatted at 2026-08-09 02:12:14 on dist-test-slave-p3t5"
server_key: "0e65b716bb9285986b8de94f38c6c09c"
server_key_iv: "e7eee5304c64648134675841bd2863b9"
server_key_version: "kuduclusterkey@0"
I20260809 02:12:14.581414 18381 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.002s
I20260809 02:12:14.606755 18397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:14.607085 18381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 02:12:14.607200 18381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "3721555f5e494cb39912b96f4e3622bb"
format_stamp: "Formatted at 2026-08-09 02:12:14 on dist-test-slave-p3t5"
server_key: "0e65b716bb9285986b8de94f38c6c09c"
server_key_iv: "e7eee5304c64648134675841bd2863b9"
server_key_version: "kuduclusterkey@0"
I20260809 02:12:14.607313 18381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:14.639508 18381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:14.642258 18381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:14.642558 18381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:14.642879 18381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:14.643353 18381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:12:14.643461 18381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:14.643545 18381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:12:14.643608 18381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:14.653023 18381 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:33233
I20260809 02:12:14.653674 18381 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:12:14.655632 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 18381
I20260809 02:12:14.655750 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:12:14.655972 5158 external_mini_cluster.cc:1428] Setting key 244f9d3c91b8afb241a7c36512eceab6
I20260809 02:12:14.654470 18513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:33233 every 8 connection(s)
Aug 09 02:12:14 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241534, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:14.667635 18235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:14.656262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:45547 (local address 127.5.9.190:33853)
0809 02:12:14.656326 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:14.656328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:14.656340 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:12:14.656830 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:14.656832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:14.656846 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:14.656879 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:14.657841 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:14.658652 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:14.660194 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:14.660519 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:14.660569 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:14.663859 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:14.663861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:14.663861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:14.663870 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:14.665751 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:14.666329 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:14.666330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:14.666331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:14.666396 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:14.666754 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:14.666755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:14.666756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:14.666917 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:14.666966 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:14.667406 (+ 440us) server_negotiation.cc:299] Negotiation successful
0809 02:12:14.667529 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:12:14.668521 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:14.655106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:33853 (local address 127.5.9.131:45547)
0809 02:12:14.655597 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:14.655646 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:14.656691 (+ 1045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:14.657090 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:14.657094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:14.657371 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:14.657718 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:14.657726 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:14.658782 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:14.658783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:14.660083 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:14.660086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:14.660695 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:14.660695 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:14.660855 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:14.661836 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:14.661847 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:14.663730 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:14.665871 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:14.665882 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:14.665885 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:14.666233 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:14.666486 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:14.666487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:14.666487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:14.666644 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:14.667020 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:14.667028 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:14.667243 (+ 215us) client_negotiation.cc:769] Sending connection context
0809 02:12:14.668049 (+ 806us) client_negotiation.cc:240] Negotiation successful
0809 02:12:14.668244 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":82,"threads_started":1}
I20260809 02:12:14.669332 18515 heartbeater.cc:347] Connected to a master server at 127.5.9.190:33853
I20260809 02:12:14.669447 18515 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:14.669673 18515 heartbeater.cc:511] Master 127.5.9.190:33853 requested a full tablet report, sending...
I20260809 02:12:14.670130 18002 ts_manager.cc:194] Registered new tserver with Master: 3721555f5e494cb39912b96f4e3622bb (127.5.9.131:33233)
I20260809 02:12:14.670663 18002 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:45547
I20260809 02:12:14.672412 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:12:14.673177 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 18095
I20260809 02:12:14.696166 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 18240
Aug 09 02:12:14 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241534, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
I20260809 02:12:14.716794 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 18381
I20260809 02:12:14.727502 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 17978
I20260809 02:12:14.737032 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:33853
--webserver_interface=127.5.9.190
--webserver_port=45941
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:33853
--ranger_config_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.9.148:46207/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.9.148:46207/kms with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:14.826244 18518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:14.826498 18518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:14.826524 18518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:14.827843 18518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:12:14.827898 18518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:14.827912 18518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:14.827925 18518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:12:14.827946 18518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:12:14.829811 18518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.9.148:46207/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:33853
--ranger_config_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:33853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=45941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:14.830096 18518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:14.830369 18518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:14.836086 18526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:14.836102 18523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:14.836135 18524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:14.836560 18518 server_base.cc:1034] running on GCE node
I20260809 02:12:14.836875 18518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:14.837239 18518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:14.838476 18518 hybrid_clock.cc:648] HybridClock initialized: now 1786241534838457 us; error 34 us; skew 500 ppm
Aug 09 02:12:14 dist-test-slave-p3t5 krb5kdc[17511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241534, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:14.841423 18518 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:12:14.842069 18518 webserver.cc:460] Webserver started at http://127.5.9.190:45941/ using document root <none> and password file <none>
I20260809 02:12:14.842329 18518 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:14.842404 18518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 02:12:14 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241534, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
I20260809 02:12:14.926409 18518 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:12:14.927258 18534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:14.927459 18518 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:12:14.927562 18518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "284c0350e1b24f508c56744bc7cbff44"
format_stamp: "Formatted at 2026-08-09 02:12:12 on dist-test-slave-p3t5"
server_key: "065824d07b0a29b69959fffba84047c8"
server_key_iv: "cf166239ba59a58bdf5fd2ad89533b39"
server_key_version: "kuduclusterkey@0"
I20260809 02:12:14.927979 18518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:14.943899 18518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:14.946144 18518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:14.946302 18518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:14.951538 18518 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:33853
I20260809 02:12:14.951582 18586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:33853 every 8 connection(s)
I20260809 02:12:14.952018 18518 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:12:14.953377 18587 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:12:14.954180 18587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44: Bootstrap starting.
I20260809 02:12:14.955809 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 18518
I20260809 02:12:14.956141 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.9.148:46207/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:44433
--local_ip_for_outbound_sockets=127.5.9.129
--tserver_master_addrs=127.5.9.190:33853
--webserver_port=33113
--webserver_interface=127.5.9.129
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.9.148:46207/kms with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:12:14.965849 18587 log.cc:824] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:14.968736 18587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:12:14.969156 18587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44: Bootstrap complete.
I20260809 02:12:14.973197 18587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284c0350e1b24f508c56744bc7cbff44" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33853 } }
I20260809 02:12:14.973575 18587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284c0350e1b24f508c56744bc7cbff44, State: Initialized, Role: FOLLOWER
I20260809 02:12:14.973840 18587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284c0350e1b24f508c56744bc7cbff44" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33853 } }
I20260809 02:12:14.973924 18587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:12:14.973969 18587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:12:14.974011 18587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:12:14.975519 18587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284c0350e1b24f508c56744bc7cbff44" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33853 } }
I20260809 02:12:14.975657 18587 leader_election.cc:302] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 284c0350e1b24f508c56744bc7cbff44; no voters:
I20260809 02:12:14.975857 18587 leader_election.cc:288] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 02:12:14.975967 18592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:12:14.976142 18587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:12:14.976184 18592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [term 2 LEADER]: Becoming Leader. State: Replica: 284c0350e1b24f508c56744bc7cbff44, State: Running, Role: LEADER
I20260809 02:12:14.976385 18592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284c0350e1b24f508c56744bc7cbff44" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33853 } }
I20260809 02:12:14.976722 18593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "284c0350e1b24f508c56744bc7cbff44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284c0350e1b24f508c56744bc7cbff44" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33853 } } }
I20260809 02:12:14.976818 18593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [sys.catalog]: This master's current role is: LEADER
I20260809 02:12:14.977108 18594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 284c0350e1b24f508c56744bc7cbff44. Latest consensus state: current_term: 2 leader_uuid: "284c0350e1b24f508c56744bc7cbff44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284c0350e1b24f508c56744bc7cbff44" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33853 } } }
I20260809 02:12:14.977192 18594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44 [sys.catalog]: This master's current role is: LEADER
I20260809 02:12:14.977279 18602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:12:14.977764 18587 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 02:12:14.977938 18602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:12:14.978142 18602 catalog_manager.cc:1259] Loaded cluster ID: da3883d463bb449791de6f54ab13bab1
I20260809 02:12:14.978214 18602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:12:14.983194 18602 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:12:14.983726 18602 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44: Loaded TSK: 0
I20260809 02:12:14.984148 18602 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:12:15.060551 18590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:15.060948 18590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:15.061010 18590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:15.062918 18590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:15.062988 18590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:15.063035 18590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:12:15.065586 18590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.9.148:46207/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:44433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=33113
--tserver_master_addrs=127.5.9.190:33853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:15.065917 18590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:15.066237 18590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:15.073359 18629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:15.073417 18630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:15.074150 18632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:15.074425 18590 server_base.cc:1034] running on GCE node
I20260809 02:12:15.074661 18590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:15.074985 18590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:15.076174 18590 hybrid_clock.cc:648] HybridClock initialized: now 1786241535076141 us; error 52 us; skew 500 ppm
Aug 09 02:12:15 dist-test-slave-p3t5 krb5kdc[17511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241535, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:15.079736 18590 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:12:15.080376 18590 webserver.cc:460] Webserver started at http://127.5.9.129:33113/ using document root <none> and password file <none>
I20260809 02:12:15.080655 18590 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:15.080742 18590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 02:12:15 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241535, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
I20260809 02:12:15.110237 18590 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:15.111438 18640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:15.111610 18590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:15.111693 18590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "3e6017ba692c44e4a710ee050b629392"
format_stamp: "Formatted at 2026-08-09 02:12:13 on dist-test-slave-p3t5"
server_key: "387ac71243d53e7d6fa32ed88e948b75"
server_key_iv: "568eff03226691ec904bf72166257e1d"
server_key_version: "kuduclusterkey@0"
I20260809 02:12:15.112205 18590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:15.135638 18590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:15.138008 18590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:15.138197 18590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:15.138474 18590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:15.138871 18590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:12:15.138928 18590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:15.138970 18590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:12:15.138993 18590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:15.146430 18590 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:44433
I20260809 02:12:15.146934 18590 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:12:15.148816 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 18590
I20260809 02:12:15.149132 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.9.148:46207/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:42321
--local_ip_for_outbound_sockets=127.5.9.130
--tserver_master_addrs=127.5.9.190:33853
--webserver_port=33931
--webserver_interface=127.5.9.130
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.9.148:46207/kms with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:12:15.154440 18760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:44433 every 8 connection(s)
Aug 09 02:12:15 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241535, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:15.180034 18764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:15.154740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:33853 (local address 127.5.9.129:46353)
0809 02:12:15.164296 (+ 9556us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:15.164380 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:15.165881 (+ 1501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:15.166628 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:15.166632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:15.166944 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:15.167255 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:15.167263 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:15.169061 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:15.169062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:15.170387 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:15.170390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:15.171789 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:15.171790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:15.171952 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:15.172645 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:15.172657 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:15.174476 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:15.177032 (+ 2556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:15.177041 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:15.177044 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:15.177435 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:15.177782 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:15.177783 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:15.177783 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:15.177994 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:15.179277 (+ 1283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:15.179285 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:15.179461 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 02:12:15.179569 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 02:12:15.179757 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9485,"thread_start_us":129,"threads_started":1}
I20260809 02:12:15.190558 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:15.155193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:46353 (local address 127.5.9.190:33853)
0809 02:12:15.164944 (+ 9751us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:15.164955 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:15.165014 (+ 59us) server_negotiation.cc:407] Connection header received
0809 02:12:15.166236 (+ 1222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:15.166244 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:15.166312 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:15.166452 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:15.167405 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:15.168923 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:15.170495 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:15.171378 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:15.171497 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:15.174616 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:15.174620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:15.174621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:15.174633 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:15.176931 (+ 2298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:15.177538 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:15.177541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:15.177542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:15.177696 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:15.178079 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:15.178080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:15.178081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:15.179151 (+ 1070us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:15.179245 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:15.180874 (+ 1629us) server_negotiation.cc:299] Negotiation successful
0809 02:12:15.181275 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9679,"thread_start_us":101,"threads_started":1}
I20260809 02:12:15.191936 18761 heartbeater.cc:347] Connected to a master server at 127.5.9.190:33853
I20260809 02:12:15.192054 18761 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:15.192276 18761 heartbeater.cc:511] Master 127.5.9.190:33853 requested a full tablet report, sending...
W20260809 02:12:15.193306 18761 heartbeater.cc:644] Failed to heartbeat to 127.5.9.190:33853 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 02:12:15.193396 18761 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:15.193480 18761 heartbeater.cc:511] Master 127.5.9.190:33853 requested a full tablet report, sending...
I20260809 02:12:15.193876 18761 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:15.193969 18761 heartbeater.cc:511] Master 127.5.9.190:33853 requested a full tablet report, sending...
W20260809 02:12:15.194314 18761 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:12:15.263341 18763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:15.263640 18763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:15.263684 18763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:15.264919 18763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:15.264995 18763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:15.265043 18763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:12:15.267350 18763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.9.148:46207/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:42321
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=33931
--tserver_master_addrs=127.5.9.190:33853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:15.267608 18763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:15.267882 18763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:15.274250 18774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:15.274573 18770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:15.274781 18771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:15.275194 18763 server_base.cc:1034] running on GCE node
I20260809 02:12:15.275437 18763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:15.275722 18763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:15.276906 18763 hybrid_clock.cc:648] HybridClock initialized: now 1786241535276873 us; error 46 us; skew 500 ppm
Aug 09 02:12:15 dist-test-slave-p3t5 krb5kdc[17511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241535, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:15.280205 18763 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:12:15.280850 18763 webserver.cc:460] Webserver started at http://127.5.9.130:33931/ using document root <none> and password file <none>
I20260809 02:12:15.281095 18763 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:15.281185 18763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 02:12:15 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241535, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
I20260809 02:12:15.298038 18763 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:15.299176 18782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:15.299515 18763 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:15.299631 18763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "011d00facd21485d82152593a6c28345"
format_stamp: "Formatted at 2026-08-09 02:12:14 on dist-test-slave-p3t5"
server_key: "0b3ee305a6a9922146ca25680c795d4c"
server_key_iv: "e7d874bcd9752c44abba6345ad7d9970"
server_key_version: "kuduclusterkey@0"
I20260809 02:12:15.300155 18763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:15.320243 18763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:15.322814 18763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:15.322992 18763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:15.323271 18763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:15.323688 18763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:12:15.323738 18763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:15.323772 18763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:12:15.323789 18763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:15.331715 18763 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:42321
I20260809 02:12:15.331924 18900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:42321 every 8 connection(s)
I20260809 02:12:15.332275 18763 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:12:15 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241535, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:15.339998 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 18763
I20260809 02:12:15.340322 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.9.148:46207/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:33233
--local_ip_for_outbound_sockets=127.5.9.131
--tserver_master_addrs=127.5.9.190:33853
--webserver_port=36795
--webserver_interface=127.5.9.131
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.9.148:46207/kms with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:12:15.343405 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:15.333140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:49421 (local address 127.5.9.190:33853)
0809 02:12:15.333192 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:15.333194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:15.333469 (+ 275us) server_negotiation.cc:407] Connection header received
0809 02:12:15.334222 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:15.334223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:15.334235 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:15.334296 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:15.335038 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:15.335540 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:15.337064 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:15.337298 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:15.337368 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:15.339977 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:15.339979 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:15.339979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:15.339995 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:15.341670 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:15.342130 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:15.342132 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:15.342132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:15.342202 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:15.342543 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:15.342544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:15.342545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:15.342724 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:15.342815 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:15.343197 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 02:12:15.343301 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:12:15.343634 18903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:15.333124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:33853 (local address 127.5.9.130:49421)
0809 02:12:15.333293 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:15.333345 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:15.334093 (+ 748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:15.334379 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:15.334384 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:15.334639 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:15.334942 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:15.334949 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:15.335651 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:15.335652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:15.336962 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:15.336966 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:15.337464 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:15.337465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:15.337609 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:15.338360 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:15.338371 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:15.339859 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:15.341777 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:15.341786 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:15.341789 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:15.342043 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:15.342281 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:15.342282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:15.342282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:15.342452 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:15.342816 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:15.342823 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:15.343016 (+ 193us) client_negotiation.cc:769] Sending connection context
0809 02:12:15.343144 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 02:12:15.343340 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260809 02:12:15.344134 18901 heartbeater.cc:347] Connected to a master server at 127.5.9.190:33853
I20260809 02:12:15.344237 18901 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:15.344455 18901 heartbeater.cc:511] Master 127.5.9.190:33853 requested a full tablet report, sending...
W20260809 02:12:15.345201 18901 heartbeater.cc:644] Failed to heartbeat to 127.5.9.190:33853 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 02:12:15.345288 18901 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:15.345377 18901 heartbeater.cc:511] Master 127.5.9.190:33853 requested a full tablet report, sending...
I20260809 02:12:15.345731 18901 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:15.345814 18901 heartbeater.cc:511] Master 127.5.9.190:33853 requested a full tablet report, sending...
W20260809 02:12:15.346119 18901 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:12:15.449239 18904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:15.449563 18904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:15.449605 18904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:15.451479 18904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:15.451567 18904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:15.451615 18904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:12:15.454048 18904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.9.148:46207/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:33233
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=36795
--tserver_master_addrs=127.5.9.190:33853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:15.454355 18904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:15.454648 18904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:15.461129 18910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:15.461480 18904 server_base.cc:1034] running on GCE node
W20260809 02:12:15.461755 18909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:15.463181 18912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:15.463505 18904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:15.463809 18904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:15.464998 18904 hybrid_clock.cc:648] HybridClock initialized: now 1786241535464958 us; error 54 us; skew 500 ppm
Aug 09 02:12:15 dist-test-slave-p3t5 krb5kdc[17511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241535, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:15.468551 18904 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:12:15.469112 18904 webserver.cc:460] Webserver started at http://127.5.9.131:36795/ using document root <none> and password file <none>
I20260809 02:12:15.469419 18904 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:15.469498 18904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 02:12:15 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241535, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
I20260809 02:12:15.497857 18904 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.000s
I20260809 02:12:15.498981 18921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:15.499168 18904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:15.499308 18904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "3721555f5e494cb39912b96f4e3622bb"
format_stamp: "Formatted at 2026-08-09 02:12:14 on dist-test-slave-p3t5"
server_key: "0e65b716bb9285986b8de94f38c6c09c"
server_key_iv: "e7eee5304c64648134675841bd2863b9"
server_key_version: "kuduclusterkey@0"
I20260809 02:12:15.499843 18904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:15.515111 18904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:15.527189 18904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:15.527410 18904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:15.527657 18904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:15.528082 18904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:12:15.528158 18904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:15.528196 18904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:12:15.528219 18904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:15.536248 18904 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:33233
I20260809 02:12:15.536777 18904 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:12:15.537635 19039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:33233 every 8 connection(s)
I20260809 02:12:15.538846 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 18904
Aug 09 02:12:15 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241535, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:15.557652 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:15.538368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:42157 (local address 127.5.9.190:33853)
0809 02:12:15.538479 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:15.538482 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:15.545067 (+ 6585us) server_negotiation.cc:407] Connection header received
0809 02:12:15.545998 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:15.545999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:15.546014 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:15.546047 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:15.547042 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:15.547812 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:15.549404 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:15.549728 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:15.549789 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:15.553451 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:15.553452 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:15.553452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:15.553460 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:15.555346 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:15.556425 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:15.556427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:15.556427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:15.556491 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:15.556786 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:15.556787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:15.556788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:15.556929 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:15.556977 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:15.557428 (+ 451us) server_negotiation.cc:299] Negotiation successful
0809 02:12:15.557527 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:12:15.558220 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:15.538338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:33853 (local address 127.5.9.131:42157)
0809 02:12:15.544850 (+ 6512us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:15.544964 (+ 114us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:15.545852 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:15.546272 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:15.546278 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:15.546565 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:15.546924 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:15.546931 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:15.547947 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:15.547948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:15.549293 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:15.549297 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:15.549917 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:15.549918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:15.550077 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:15.550789 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:15.550800 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:15.553334 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:15.555968 (+ 2634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:15.555978 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:15.555981 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:15.556334 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:15.556559 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:15.556560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:15.556560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:15.556695 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:15.557032 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:15.557040 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:15.557247 (+ 207us) client_negotiation.cc:769] Sending connection context
0809 02:12:15.557758 (+ 511us) client_negotiation.cc:240] Negotiation successful
0809 02:12:15.557964 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6401,"thread_start_us":115,"threads_started":1}
I20260809 02:12:15.558748 19040 heartbeater.cc:347] Connected to a master server at 127.5.9.190:33853
I20260809 02:12:15.558866 19040 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:15.559072 19040 heartbeater.cc:511] Master 127.5.9.190:33853 requested a full tablet report, sending...
W20260809 02:12:15.559674 19040 heartbeater.cc:644] Failed to heartbeat to 127.5.9.190:33853 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 02:12:15.559751 19040 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:15.559829 19040 heartbeater.cc:511] Master 127.5.9.190:33853 requested a full tablet report, sending...
I20260809 02:12:15.560120 19040 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:15.560197 19040 heartbeater.cc:511] Master 127.5.9.190:33853 requested a full tablet report, sending...
W20260809 02:12:15.560467 19040 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 02:12:15.561512 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:15.539301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52232 (local address 127.5.9.190:33853)
0809 02:12:15.544304 (+ 5003us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:15.544307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:15.544327 (+ 20us) server_negotiation.cc:407] Connection header received
0809 02:12:15.544414 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:15.544416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:15.544435 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:15.544609 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:15.552144 (+ 7535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:15.552930 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:15.555421 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:15.555743 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:15.555868 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:15.557692 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:15.557694 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:15.557694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:15.557703 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:15.560086 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:15.560577 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:15.560578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:15.560579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:15.560649 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:15.560892 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:15.560893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:15.560893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:15.561042 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:15.561235 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:15.561315 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 02:12:15.561407 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4941,"thread_start_us":83,"threads_started":1}
Aug 09 02:12:16 dist-test-slave-p3t5 krb5kdc[17511](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241536, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:16.194554 18761 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:16.194782 18761 heartbeater.cc:511] Master 127.5.9.190:33853 requested a full tablet report, sending...
I20260809 02:12:16.195267 18551 ts_manager.cc:194] Registered new tserver with Master: 3e6017ba692c44e4a710ee050b629392 (127.5.9.129:44433)
I20260809 02:12:16.196029 18551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:46353
I20260809 02:12:16.346329 18901 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:16.346534 18901 heartbeater.cc:511] Master 127.5.9.190:33853 requested a full tablet report, sending...
I20260809 02:12:16.346933 18551 ts_manager.cc:194] Registered new tserver with Master: 011d00facd21485d82152593a6c28345 (127.5.9.130:42321)
I20260809 02:12:16.347368 18551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:49421
I20260809 02:12:16.560653 19040 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:16.560835 19040 heartbeater.cc:511] Master 127.5.9.190:33853 requested a full tablet report, sending...
I20260809 02:12:16.562772 18551 ts_manager.cc:194] Registered new tserver with Master: 3721555f5e494cb39912b96f4e3622bb (127.5.9.131:33233)
I20260809 02:12:16.563309 18551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:42157
I20260809 02:12:16.572338 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:12:16.585843 19063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:16.573391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52248 (local address 127.5.9.190:33853)
0809 02:12:16.573532 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:16.573535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:16.574014 (+ 479us) server_negotiation.cc:407] Connection header received
0809 02:12:16.574102 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:16.574103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:16.574116 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:16.574187 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:16.574886 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.575679 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.576904 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.577238 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.577651 (+ 413us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:16.581903 (+ 4252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:16.581905 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:16.581905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:16.581915 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:16.583863 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:16.584453 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:16.584455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:16.584455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:16.584538 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:16.585084 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:16.585085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:16.585085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:16.585226 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:16.585351 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:16.585494 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 02:12:16.585596 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
I20260809 02:12:16.592188 18551 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:12:16.594894 18551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:12:16.610298 19069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:16.602890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:33233 (local address 127.0.0.1:39174)
0809 02:12:16.603627 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:16.603670 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:16.603982 (+ 312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:16.604350 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:16.604353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:16.604357 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:16.604432 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:16.604434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.606426 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.606427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:16.608627 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:16.608632 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.609964 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.609965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:16.610051 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:16.610065 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 02:12:16.610137 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:12:16.610199 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":87,"threads_started":1}
I20260809 02:12:16.610350 19073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:16.603160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39174 (local address 127.5.9.131:33233)
0809 02:12:16.603438 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:16.603444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:16.603734 (+ 290us) server_negotiation.cc:407] Connection header received
0809 02:12:16.604110 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:16.604113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:16.604131 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:16.604186 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:16.604517 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.606316 (+ 1799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.608758 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.609768 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.609829 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:16.609897 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:16.610198 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 02:12:16.610258 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":48,"threads_started":1}
Aug 09 02:12:16 dist-test-slave-p3t5 krb5kdc[17511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241536, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for HTTP/127.5.9.148@KRBTEST.COM
I20260809 02:12:16.614902 18972 tablet_service.cc:1467] Processing CreateTablet for tablet 31dcad175e2c4c048d50e37835946d77 (DEFAULT_TABLE table=test-table [id=45faa37f22334047961fc1670ecb9b23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:12:16.615366 19071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:16.602985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56802 (local address 127.5.9.129:44433)
0809 02:12:16.603567 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:16.603571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:16.611245 (+ 7674us) server_negotiation.cc:407] Connection header received
0809 02:12:16.611296 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:16.611299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:16.611316 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:16.611402 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:16.611647 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.612567 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.613719 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.614785 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.614849 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:16.614908 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:16.615234 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 02:12:16.615283 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":52,"threads_started":1}
I20260809 02:12:16.615381 18972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31dcad175e2c4c048d50e37835946d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:16.615767 19070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:16.602890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:44433 (local address 127.0.0.1:56802)
0809 02:12:16.610774 (+ 7884us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:16.610788 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:16.610871 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:16.611508 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:16.611509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:16.611511 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:16.611557 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:16.611559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.612669 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.612670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:16.613583 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:16.613586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.614939 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.614940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:16.615001 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:16.615005 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:12:16.615049 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 02:12:16.615093 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7860,"thread_start_us":120,"threads_started":1}
I20260809 02:12:16.616391 19072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:16.603135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:42321 (local address 127.0.0.1:44618)
0809 02:12:16.610947 (+ 7812us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:16.610955 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:16.610992 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:16.611290 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:16.611292 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:16.611294 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:16.611338 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:16.611340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.614072 (+ 2732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.614073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:16.615098 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:16.615100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.616156 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.616157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:16.616220 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:16.616225 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:12:16.616282 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 02:12:16.616328 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7799,"thread_start_us":66,"threads_started":1}
I20260809 02:12:16.616811 19074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:16.603392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44618 (local address 127.5.9.130:42321)
0809 02:12:16.607801 (+ 4409us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:16.607808 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:16.611051 (+ 3243us) server_negotiation.cc:407] Connection header received
0809 02:12:16.611105 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:16.611108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:16.611125 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:16.611196 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:16.611410 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.613968 (+ 2558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.615190 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.616007 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.616085 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:16.616629 (+ 544us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:16.616659 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 02:12:16.616712 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4365,"thread_start_us":62,"threads_started":1}
I20260809 02:12:16.617494 18815 tablet_service.cc:1467] Processing CreateTablet for tablet 31dcad175e2c4c048d50e37835946d77 (DEFAULT_TABLE table=test-table [id=45faa37f22334047961fc1670ecb9b23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:12:16.617929 18815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31dcad175e2c4c048d50e37835946d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:16.619951 18692 tablet_service.cc:1467] Processing CreateTablet for tablet 31dcad175e2c4c048d50e37835946d77 (DEFAULT_TABLE table=test-table [id=45faa37f22334047961fc1670ecb9b23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:12:16.620302 18692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31dcad175e2c4c048d50e37835946d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:16.622017 19075 tablet_bootstrap.cc:492] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb: Bootstrap starting.
I20260809 02:12:16.622596 19076 tablet_bootstrap.cc:492] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345: Bootstrap starting.
I20260809 02:12:16.626346 19076 tablet_bootstrap.cc:654] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:16.627189 19075 tablet_bootstrap.cc:654] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:16.627619 19076 log.cc:824] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:16.628515 19075 log.cc:824] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:16.629186 19077 tablet_bootstrap.cc:492] T 31dcad175e2c4c048d50e37835946d77 P 3e6017ba692c44e4a710ee050b629392: Bootstrap starting.
I20260809 02:12:16.629994 19077 tablet_bootstrap.cc:654] T 31dcad175e2c4c048d50e37835946d77 P 3e6017ba692c44e4a710ee050b629392: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:16.630378 19077 log.cc:824] T 31dcad175e2c4c048d50e37835946d77 P 3e6017ba692c44e4a710ee050b629392: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:16.633757 19076 tablet_bootstrap.cc:492] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345: No bootstrap required, opened a new log
I20260809 02:12:16.633858 19076 ts_tablet_manager.cc:1397] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260809 02:12:16.636425 19076 raft_consensus.cc:348] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6017ba692c44e4a710ee050b629392" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 44433 } } peers { permanent_uuid: "011d00facd21485d82152593a6c28345" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 42321 } } peers { permanent_uuid: "3721555f5e494cb39912b96f4e3622bb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 33233 } }
I20260809 02:12:16.636656 19076 raft_consensus.cc:374] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:16.636714 19076 raft_consensus.cc:729] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011d00facd21485d82152593a6c28345, State: Initialized, Role: FOLLOWER
I20260809 02:12:16.636888 19076 consensus_queue.cc:260] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6017ba692c44e4a710ee050b629392" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 44433 } } peers { permanent_uuid: "011d00facd21485d82152593a6c28345" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 42321 } } peers { permanent_uuid: "3721555f5e494cb39912b96f4e3622bb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 33233 } }
I20260809 02:12:16.637250 19076 ts_tablet_manager.cc:1428] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:12:16.637401 18901 heartbeater.cc:503] Master 127.5.9.190:33853 was elected leader, sending a full tablet report...
I20260809 02:12:16.638623 19075 tablet_bootstrap.cc:492] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb: No bootstrap required, opened a new log
I20260809 02:12:16.638731 19075 ts_tablet_manager.cc:1397] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20260809 02:12:16.642524 19075 raft_consensus.cc:348] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6017ba692c44e4a710ee050b629392" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 44433 } } peers { permanent_uuid: "011d00facd21485d82152593a6c28345" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 42321 } } peers { permanent_uuid: "3721555f5e494cb39912b96f4e3622bb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 33233 } }
I20260809 02:12:16.642843 19075 raft_consensus.cc:374] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:16.642918 19075 raft_consensus.cc:729] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3721555f5e494cb39912b96f4e3622bb, State: Initialized, Role: FOLLOWER
I20260809 02:12:16.643138 19075 consensus_queue.cc:260] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6017ba692c44e4a710ee050b629392" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 44433 } } peers { permanent_uuid: "011d00facd21485d82152593a6c28345" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 42321 } } peers { permanent_uuid: "3721555f5e494cb39912b96f4e3622bb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 33233 } }
I20260809 02:12:16.643666 19075 ts_tablet_manager.cc:1428] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:12:16.643810 19040 heartbeater.cc:503] Master 127.5.9.190:33853 was elected leader, sending a full tablet report...
I20260809 02:12:16.646162 19077 tablet_bootstrap.cc:492] T 31dcad175e2c4c048d50e37835946d77 P 3e6017ba692c44e4a710ee050b629392: No bootstrap required, opened a new log
I20260809 02:12:16.646275 19077 ts_tablet_manager.cc:1397] T 31dcad175e2c4c048d50e37835946d77 P 3e6017ba692c44e4a710ee050b629392: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20260809 02:12:16.648922 19077 raft_consensus.cc:348] T 31dcad175e2c4c048d50e37835946d77 P 3e6017ba692c44e4a710ee050b629392 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6017ba692c44e4a710ee050b629392" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 44433 } } peers { permanent_uuid: "011d00facd21485d82152593a6c28345" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 42321 } } peers { permanent_uuid: "3721555f5e494cb39912b96f4e3622bb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 33233 } }
I20260809 02:12:16.649163 19077 raft_consensus.cc:374] T 31dcad175e2c4c048d50e37835946d77 P 3e6017ba692c44e4a710ee050b629392 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:16.649222 19077 raft_consensus.cc:729] T 31dcad175e2c4c048d50e37835946d77 P 3e6017ba692c44e4a710ee050b629392 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e6017ba692c44e4a710ee050b629392, State: Initialized, Role: FOLLOWER
I20260809 02:12:16.649389 19077 consensus_queue.cc:260] T 31dcad175e2c4c048d50e37835946d77 P 3e6017ba692c44e4a710ee050b629392 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6017ba692c44e4a710ee050b629392" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 44433 } } peers { permanent_uuid: "011d00facd21485d82152593a6c28345" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 42321 } } peers { permanent_uuid: "3721555f5e494cb39912b96f4e3622bb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 33233 } }
I20260809 02:12:16.649778 19077 ts_tablet_manager.cc:1428] T 31dcad175e2c4c048d50e37835946d77 P 3e6017ba692c44e4a710ee050b629392: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260809 02:12:16.649931 18761 heartbeater.cc:503] Master 127.5.9.190:33853 was elected leader, sending a full tablet report...
W20260809 02:12:16.655099 18762 tablet.cc:2369] T 31dcad175e2c4c048d50e37835946d77 P 3e6017ba692c44e4a710ee050b629392: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:12:16.746376 19081 raft_consensus.cc:482] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:12:16.746564 19081 raft_consensus.cc:504] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6017ba692c44e4a710ee050b629392" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 44433 } } peers { permanent_uuid: "011d00facd21485d82152593a6c28345" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 42321 } } peers { permanent_uuid: "3721555f5e494cb39912b96f4e3622bb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 33233 } }
I20260809 02:12:16.746997 19081 leader_election.cc:288] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e6017ba692c44e4a710ee050b629392 (127.5.9.129:44433), 3721555f5e494cb39912b96f4e3622bb (127.5.9.131:33233)
I20260809 02:12:16.751680 19073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:16.747597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:51299 (local address 127.5.9.131:33233)
0809 02:12:16.747834 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:16.747836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:16.747843 (+ 7us) server_negotiation.cc:407] Connection header received
0809 02:12:16.747871 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:16.747872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:16.747884 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:16.747925 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:16.748177 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.748890 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.750514 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.751075 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.751127 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:16.751140 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:16.751550 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 02:12:16.751600 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215}
I20260809 02:12:16.752030 19084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:16.747215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:33233 (local address 127.5.9.130:51299)
0809 02:12:16.747666 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:16.747678 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:16.747740 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:16.748014 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:16.748015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:16.748017 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:16.748103 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:16.748105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.748987 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.748987 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:16.750384 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:16.750388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.751308 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.751309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:16.751450 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:16.751455 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:12:16.751817 (+ 362us) client_negotiation.cc:240] Negotiation successful
0809 02:12:16.751861 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":114,"threads_started":1}
I20260809 02:12:16.752363 18992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31dcad175e2c4c048d50e37835946d77" candidate_uuid: "011d00facd21485d82152593a6c28345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3721555f5e494cb39912b96f4e3622bb" is_pre_election: true
I20260809 02:12:16.752548 18992 raft_consensus.cc:2454] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 011d00facd21485d82152593a6c28345 in term 0.
I20260809 02:12:16.752803 18786 leader_election.cc:302] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011d00facd21485d82152593a6c28345, 3721555f5e494cb39912b96f4e3622bb; no voters:
I20260809 02:12:16.752933 19071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:16.747527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:43699 (local address 127.5.9.129:44433)
0809 02:12:16.747985 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:16.747987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:16.748506 (+ 519us) server_negotiation.cc:407] Connection header received
0809 02:12:16.748616 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:16.748618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:16.748632 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:16.748668 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:16.749165 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.749901 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.751432 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.752073 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.752123 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:16.752133 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:16.752802 (+ 669us) server_negotiation.cc:299] Negotiation successful
0809 02:12:16.752862 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435}
I20260809 02:12:16.752941 19081 raft_consensus.cc:2781] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:12:16.752980 19081 raft_consensus.cc:482] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:12:16.753002 19081 raft_consensus.cc:3037] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:16.753350 19085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:16.747215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:44433 (local address 127.5.9.130:43699)
0809 02:12:16.748423 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:16.748437 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:16.748537 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:16.748741 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:16.748742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:16.748744 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:16.749043 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:16.749046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.750014 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.750015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:16.751326 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:16.751330 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.752585 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.752586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:16.752716 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:16.752719 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:12:16.753214 (+ 495us) client_negotiation.cc:240] Negotiation successful
0809 02:12:16.753263 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1188,"mutex_wait_us":16,"thread_start_us":114,"threads_started":1}
I20260809 02:12:16.753942 18708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31dcad175e2c4c048d50e37835946d77" candidate_uuid: "011d00facd21485d82152593a6c28345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e6017ba692c44e4a710ee050b629392" is_pre_election: true
I20260809 02:12:16.754168 18708 raft_consensus.cc:2454] T 31dcad175e2c4c048d50e37835946d77 P 3e6017ba692c44e4a710ee050b629392 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 011d00facd21485d82152593a6c28345 in term 0.
I20260809 02:12:16.754262 19081 raft_consensus.cc:504] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6017ba692c44e4a710ee050b629392" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 44433 } } peers { permanent_uuid: "011d00facd21485d82152593a6c28345" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 42321 } } peers { permanent_uuid: "3721555f5e494cb39912b96f4e3622bb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 33233 } }
I20260809 02:12:16.754422 19081 leader_election.cc:288] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [CANDIDATE]: Term 1 election: Requested vote from peers 3e6017ba692c44e4a710ee050b629392 (127.5.9.129:44433), 3721555f5e494cb39912b96f4e3622bb (127.5.9.131:33233)
I20260809 02:12:16.754681 18708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31dcad175e2c4c048d50e37835946d77" candidate_uuid: "011d00facd21485d82152593a6c28345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e6017ba692c44e4a710ee050b629392"
I20260809 02:12:16.754766 18708 raft_consensus.cc:3037] T 31dcad175e2c4c048d50e37835946d77 P 3e6017ba692c44e4a710ee050b629392 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:16.755750 18708 raft_consensus.cc:2454] T 31dcad175e2c4c048d50e37835946d77 P 3e6017ba692c44e4a710ee050b629392 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011d00facd21485d82152593a6c28345 in term 1.
I20260809 02:12:16.755962 18784 leader_election.cc:302] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011d00facd21485d82152593a6c28345, 3e6017ba692c44e4a710ee050b629392; no voters:
I20260809 02:12:16.756157 18992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31dcad175e2c4c048d50e37835946d77" candidate_uuid: "011d00facd21485d82152593a6c28345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3721555f5e494cb39912b96f4e3622bb"
I20260809 02:12:16.756248 18992 raft_consensus.cc:3037] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:16.757236 18992 raft_consensus.cc:2454] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011d00facd21485d82152593a6c28345 in term 1.
I20260809 02:12:16.757527 19081 raft_consensus.cc:2781] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:12:16.757740 19081 raft_consensus.cc:686] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [term 1 LEADER]: Becoming Leader. State: Replica: 011d00facd21485d82152593a6c28345, State: Running, Role: LEADER
I20260809 02:12:16.757848 19081 consensus_queue.cc:237] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6017ba692c44e4a710ee050b629392" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 44433 } } peers { permanent_uuid: "011d00facd21485d82152593a6c28345" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 42321 } } peers { permanent_uuid: "3721555f5e494cb39912b96f4e3622bb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 33233 } }
I20260809 02:12:16.758728 18550 catalog_manager.cc:5495] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 reported cstate change: term changed from 0 to 1, leader changed from <none> to 011d00facd21485d82152593a6c28345 (127.5.9.130). New cstate: current_term: 1 leader_uuid: "011d00facd21485d82152593a6c28345" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6017ba692c44e4a710ee050b629392" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 44433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011d00facd21485d82152593a6c28345" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3721555f5e494cb39912b96f4e3622bb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 33233 } health_report { overall_health: UNKNOWN } } }
W20260809 02:12:16.788678 19041 tablet.cc:2369] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:12:16.793653 19074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:16.790215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44630 (local address 127.5.9.130:42321)
0809 02:12:16.790269 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:16.790272 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:16.790282 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:12:16.790332 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:16.790333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:16.790352 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:16.790423 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:12:16.790640 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.791360 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.792444 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:16.792725 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:16.792787 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:16.793025 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:12:16.793170 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:12:16.793244 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:16.793505 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 02:12:16.793528 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 02:12:16.796228 18708 raft_consensus.cc:1261] T 31dcad175e2c4c048d50e37835946d77 P 3e6017ba692c44e4a710ee050b629392 [term 1 FOLLOWER]: Refusing update from remote peer 011d00facd21485d82152593a6c28345: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:12:16.796243 18992 raft_consensus.cc:1261] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb [term 1 FOLLOWER]: Refusing update from remote peer 011d00facd21485d82152593a6c28345: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:12:16.796693 19086 consensus_queue.cc:1035] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e6017ba692c44e4a710ee050b629392" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 44433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:12:16.796831 19081 consensus_queue.cc:1035] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3721555f5e494cb39912b96f4e3622bb" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:12:16.800613 19089 mvcc.cc:205] Tried to move back new op lower bound from 7316445334713921536 to 7316445334560632832. Current Snapshot: MvccSnapshot[applied={T|T < 7316445334713921536}]
I20260809 02:12:16.801483 19090 mvcc.cc:205] Tried to move back new op lower bound from 7316445334713921536 to 7316445334560632832. Current Snapshot: MvccSnapshot[applied={T|T < 7316445334713921536}]
I20260809 02:12:16.803041 19091 mvcc.cc:205] Tried to move back new op lower bound from 7316445334713921536 to 7316445334560632832. Current Snapshot: MvccSnapshot[applied={T|T < 7316445334713921536 or (T in {7316445334713921536})}]
I20260809 02:12:16.805040 18551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:12:16.805155 18551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:12:16.805963 18551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 284c0350e1b24f508c56744bc7cbff44: Sending DeleteTablet for 3 replicas of tablet 31dcad175e2c4c048d50e37835946d77
I20260809 02:12:16.806500 18972 tablet_service.cc:1514] Processing DeleteTablet for tablet 31dcad175e2c4c048d50e37835946d77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:12:16 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:39174
I20260809 02:12:16.806502 18692 tablet_service.cc:1514] Processing DeleteTablet for tablet 31dcad175e2c4c048d50e37835946d77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:12:16 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:56802
I20260809 02:12:16.806612 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 18590
I20260809 02:12:16.813949 19098 tablet_replica.cc:330] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb: stopping tablet replica
I20260809 02:12:16.814146 19098 raft_consensus.cc:2229] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:12:16.814231 18815 tablet_service.cc:1514] Processing DeleteTablet for tablet 31dcad175e2c4c048d50e37835946d77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:12:16 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:44618
I20260809 02:12:16.814293 19098 raft_consensus.cc:2258] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:12:16.814519 19099 tablet_replica.cc:330] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345: stopping tablet replica
I20260809 02:12:16.814637 19099 raft_consensus.cc:2229] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:12:16.814788 19099 raft_consensus.cc:2258] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:12:16.815392 19099 ts_tablet_manager.cc:1905] T 31dcad175e2c4c048d50e37835946d77 P 011d00facd21485d82152593a6c28345: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 02:12:16.816134 18535 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:44433 (error 108)
I20260809 02:12:16.816525 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 18763
I20260809 02:12:16.818182 19072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:16.818040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:44433 (local address 127.0.0.1:56806)
0809 02:12:16.818093 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:16.818131 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:44433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 02:12:16.818343 18535 catalog_manager.cc:4584] TS 3e6017ba692c44e4a710ee050b629392 (127.5.9.129:44433): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 31dcad175e2c4c048d50e37835946d77: Network error: Client connection negotiation failed: client connection to 127.5.9.129:44433: connect: Connection refused (error 111)
I20260809 02:12:16.819028 19098 ts_tablet_manager.cc:1905] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:12:16.821036 19098 ts_tablet_manager.cc:1918] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:12:16.821197 19098 log.cc:1196] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/31dcad175e2c4c048d50e37835946d77
I20260809 02:12:16.821606 19098 ts_tablet_manager.cc:1939] T 31dcad175e2c4c048d50e37835946d77 P 3721555f5e494cb39912b96f4e3622bb: Deleting consensus metadata
I20260809 02:12:16.822311 18537 catalog_manager.cc:4841] TS 3721555f5e494cb39912b96f4e3622bb (127.5.9.131:33233): tablet 31dcad175e2c4c048d50e37835946d77 (table test-table [id=45faa37f22334047961fc1670ecb9b23]) successfully deleted
I20260809 02:12:16.824568 19072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:16.824424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:42321 (local address 127.0.0.1:44632)
0809 02:12:16.824489 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:16.824527 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.130:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 02:12:16.824860 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 18904
I20260809 02:12:16.832885 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 18518
I20260809 02:12:16.842485 5158 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 02:12:17.187495 5158 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 02:12:17.187604 5158 mini_ranger.cc:66] Stopping Ranger...
I20260809 02:12:17.532852 5158 mini_ranger.cc:68] Stopped Ranger
2026-08-09 02:12:17.533 UTC [17541] LOG: received smart shutdown request
2026-08-09 02:12:17.537 UTC [17541] LOG: background worker "logical replication launcher" (PID 17550) exited with exit code 1
2026-08-09 02:12:17.538 UTC [17545] LOG: shutting down
2026-08-09 02:12:17.730 UTC [17541] LOG: database system is shut down
2026-08-09T02:12:17Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51025 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:12:17 dist-test-slave-p3t5 krb5kdc[19110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:12:17 dist-test-slave-p3t5 krb5kdc[19110](info): set up 2 sockets
Aug 09 02:12:17 dist-test-slave-p3t5 krb5kdc[19110](info): commencing operation
krb5kdc: starting...
W20260809 02:12:18.922542 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:12:19 dist-test-slave-p3t5 krb5kdc[19110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:12:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:12:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:12:19.058596 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:44559
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:44559,127.5.9.189:34737,127.5.9.188:43429
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:19.143919 19123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:19.144205 19123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:19.144246 19123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:19.145483 19123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:12:19.145565 19123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:19.145588 19123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:19.145609 19123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:12:19.145629 19123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:12:19.147343 19123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:44559,127.5.9.189:34737,127.5.9.188:43429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:44559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:19.147650 19123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:19.147900 19123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:19.152876 19128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:19.152861 19131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:19.152849 19129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:19.153189 19123 server_base.cc:1034] running on GCE node
I20260809 02:12:19.153518 19123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:19.153868 19123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:19.155032 19123 hybrid_clock.cc:648] HybridClock initialized: now 1786241539155009 us; error 35 us; skew 500 ppm
Aug 09 02:12:19 dist-test-slave-p3t5 krb5kdc[19110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241539, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:19.157769 19123 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:12:19.158303 19123 webserver.cc:460] Webserver started at http://127.5.9.190:38715/ using document root <none> and password file <none>
I20260809 02:12:19.158536 19123 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:19.158586 19123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:19.158720 19123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:12:19.159823 19123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "94d13722d55f49dc8c00df6ba37fb14a"
format_stamp: "Formatted at 2026-08-09 02:12:19 on dist-test-slave-p3t5"
server_key: "922d3037167dff3f38d971ebf756936f"
server_key_iv: "ec36552b815abf85ed64de62bc3230ef"
server_key_version: "encryptionkey@0"
I20260809 02:12:19.160205 19123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "94d13722d55f49dc8c00df6ba37fb14a"
format_stamp: "Formatted at 2026-08-09 02:12:19 on dist-test-slave-p3t5"
server_key: "922d3037167dff3f38d971ebf756936f"
server_key_iv: "ec36552b815abf85ed64de62bc3230ef"
server_key_version: "encryptionkey@0"
I20260809 02:12:19.161590 19123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:12:19.162451 19138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:19.162642 19123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:19.162737 19123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "94d13722d55f49dc8c00df6ba37fb14a"
format_stamp: "Formatted at 2026-08-09 02:12:19 on dist-test-slave-p3t5"
server_key: "922d3037167dff3f38d971ebf756936f"
server_key_iv: "ec36552b815abf85ed64de62bc3230ef"
server_key_version: "encryptionkey@0"
I20260809 02:12:19.162844 19123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:19.177856 19123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:19.180608 19123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:19.180800 19123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:19.185397 19123 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:44559
I20260809 02:12:19.185434 19192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:44559 every 8 connection(s)
I20260809 02:12:19.185891 19123 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:12:19.186861 19193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:19.188498 19193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:12:19 dist-test-slave-p3t5 krb5kdc[19110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241539, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:19.194958 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 19123
I20260809 02:12:19.195071 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:12:19.195303 5158 external_mini_cluster.cc:1428] Setting key b8071a1d3c57d51512f35bc1dd7cb945
I20260809 02:12:19.199079 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.189385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50946 (local address 127.5.9.190:44559)
0809 02:12:19.189522 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:19.189526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:19.189555 (+ 29us) server_negotiation.cc:407] Connection header received
0809 02:12:19.190147 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:19.190149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:19.190166 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:19.190246 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:19.190871 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.191713 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.192629 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.192956 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.193039 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.195384 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:19.195396 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:19.195397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:19.195419 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:19.197109 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.197554 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.197558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.197558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.197711 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.197987 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.197988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.197988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.198363 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:19.198461 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:19.198757 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 02:12:19.198890 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
I20260809 02:12:19.199208 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.189218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:44559 (local address 127.0.0.1:50946)
0809 02:12:19.189406 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.189447 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:19.190006 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:19.190291 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:19.190293 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:19.190584 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:19.190757 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.190761 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.191834 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.191835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.192547 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.192548 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.193035 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.193036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.193096 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.193682 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:19.193699 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:19.195286 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:19.197226 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.197234 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.197237 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.197470 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.197794 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.197795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.197796 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.197910 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.198491 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:19.198500 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:19.198601 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 02:12:19.198711 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 02:12:19.199008 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":61,"thread_start_us":80,"threads_started":1}
I20260809 02:12:19.200018 19193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:12:19.200496 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.200359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:34737 (local address 127.0.0.1:33062)
0809 02:12:19.200419 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.200471 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:34737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 02:12:19.200665 19143 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.9.189:34737: connect: Connection refused (error 111)
I20260809 02:12:19.201217 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.201125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:34737 (local address 127.0.0.1:33078)
0809 02:12:19.201171 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.201199 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:34737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 02:12:19.201359 19193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.189:34737: Network error: Client connection negotiation failed: client connection to 127.5.9.189:34737: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.5.9.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.189.keytab.
Entry for principal kudu/127.5.9.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.189.keytab.
I20260809 02:12:19.225306 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.189:34737
--webserver_interface=127.5.9.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.189.keytab
--principal=kudu/127.5.9.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:44559,127.5.9.189:34737,127.5.9.188:43429
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:12:19.250617 19193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } attempt: 1
I20260809 02:12:19.251188 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.251042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:34737 (local address 127.0.0.1:33080)
0809 02:12:19.251107 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.251153 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:34737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 02:12:19.251540 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.251440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:34737 (local address 127.0.0.1:33084)
0809 02:12:19.251484 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.251511 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:34737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 02:12:19.251638 19193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.189:34737: Network error: Client connection negotiation failed: client connection to 127.5.9.189:34737: connect: Connection refused (error 111)
W20260809 02:12:19.313405 19200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:19.313760 19200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:19.313848 19200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:19.315080 19200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:12:19.315152 19200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:19.315176 19200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:19.315199 19200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:12:19.315227 19200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:12:19.316879 19200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:44559,127.5.9.189:34737,127.5.9.188:43429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.189.keytab
--principal=kudu/127.5.9.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.189:34737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.9.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:19.317186 19200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:19.317435 19200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:12:19.319778 19193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } attempt: 2
I20260809 02:12:19.320250 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.320117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:34737 (local address 127.0.0.1:33086)
0809 02:12:19.320180 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.320219 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:34737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 02:12:19.320616 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.320519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:34737 (local address 127.0.0.1:33090)
0809 02:12:19.320565 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.320595 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:34737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 02:12:19.320703 19193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.189:34737: Network error: Client connection negotiation failed: client connection to 127.5.9.189:34737: connect: Connection refused (error 111)
W20260809 02:12:19.322932 19205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:19.322938 19208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:19.322960 19206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:19.323122 19200 server_base.cc:1034] running on GCE node
I20260809 02:12:19.323459 19200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:19.323798 19200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:19.324970 19200 hybrid_clock.cc:648] HybridClock initialized: now 1786241539324958 us; error 52 us; skew 500 ppm
Aug 09 02:12:19 dist-test-slave-p3t5 krb5kdc[19110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241539, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:19.327813 19200 init.cc:377] Logged in from keytab as kudu/127.5.9.189@KRBTEST.COM (short username kudu)
I20260809 02:12:19.328243 19200 webserver.cc:460] Webserver started at http://127.5.9.189:36361/ using document root <none> and password file <none>
I20260809 02:12:19.328470 19200 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:19.328516 19200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:19.328613 19200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:12:19.329787 19200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/data/instance:
uuid: "90cd4b92c72e46418aa06483c5ae3715"
format_stamp: "Formatted at 2026-08-09 02:12:19 on dist-test-slave-p3t5"
server_key: "9d07e0dbe79bf56e28be885957fbfff9"
server_key_iv: "ff8d15ebf6edf19941c31f5897911ef9"
server_key_version: "encryptionkey@0"
I20260809 02:12:19.330120 19200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/wal/instance:
uuid: "90cd4b92c72e46418aa06483c5ae3715"
format_stamp: "Formatted at 2026-08-09 02:12:19 on dist-test-slave-p3t5"
server_key: "9d07e0dbe79bf56e28be885957fbfff9"
server_key_iv: "ff8d15ebf6edf19941c31f5897911ef9"
server_key_version: "encryptionkey@0"
I20260809 02:12:19.331519 19200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:12:19.332321 19215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:19.332547 19200 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:19.332657 19200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/wal
uuid: "90cd4b92c72e46418aa06483c5ae3715"
format_stamp: "Formatted at 2026-08-09 02:12:19 on dist-test-slave-p3t5"
server_key: "9d07e0dbe79bf56e28be885957fbfff9"
server_key_iv: "ff8d15ebf6edf19941c31f5897911ef9"
server_key_version: "encryptionkey@0"
I20260809 02:12:19.332738 19200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:19.345461 19200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:19.348055 19200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:19.348258 19200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:19.352643 19200 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.189:34737
I20260809 02:12:19.352728 19269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.189:34737 every 8 connection(s)
I20260809 02:12:19.353082 19200 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/data/info.pb
I20260809 02:12:19.354099 19270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:19.356546 19270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:12:19.362010 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 19200
I20260809 02:12:19.362133 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-1/wal/instance
I20260809 02:12:19.362350 5158 external_mini_cluster.cc:1428] Setting key b72dcaf1cdb1df440294a2737dd1d5d3
Aug 09 02:12:19 dist-test-slave-p3t5 krb5kdc[19110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241539, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.189@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.9.188@KRBTEST.COM; defaulting to no policy
I20260809 02:12:19.373229 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.357645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50962 (local address 127.5.9.190:44559)
0809 02:12:19.357703 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:19.357718 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:19.357965 (+ 247us) server_negotiation.cc:407] Connection header received
0809 02:12:19.358883 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:19.358885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:19.358900 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:19.358979 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:19.360031 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.360686 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.361922 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.362258 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.362312 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.365545 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:19.365546 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:19.365547 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:19.365557 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:19.371353 (+ 5796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.371980 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.371982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.371983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.372054 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.372374 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.372375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.372376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.372518 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:19.372593 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:19.373059 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 02:12:19.373154 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:12:19.373435 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.357568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:44559 (local address 127.0.0.1:50962)
0809 02:12:19.357805 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.357861 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:19.358665 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:19.359092 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:19.359097 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:19.359567 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:19.359920 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.359927 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.360789 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.360790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.361806 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.361809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.362443 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.362444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.362615 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.363357 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:19.363368 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:19.365418 (+ 2050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:19.371483 (+ 6065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.371501 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.371504 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.371890 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.372137 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.372138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.372138 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.372301 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.372598 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:19.372609 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:19.372869 (+ 260us) client_negotiation.cc:769] Sending connection context
0809 02:12:19.373002 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 02:12:19.373178 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":140,"threads_started":1}
I20260809 02:12:19.374229 19270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.188@KRBTEST.COM" created.
Aug 09 02:12:19 dist-test-slave-p3t5 krb5kdc[19110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241539, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.189@KRBTEST.COM for kudu/127.5.9.189@KRBTEST.COM
I20260809 02:12:19.383605 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.374549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:34737 (local address 127.0.0.1:33102)
0809 02:12:19.374603 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.374615 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:19.374666 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:19.374911 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:19.374912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:19.375095 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:19.375165 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.375168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.376063 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.376064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.376910 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.376913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.377709 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.377710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.377825 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.378334 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:19.378338 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:19.379980 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:19.382429 (+ 2449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.382430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.382431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.382623 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.382880 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.382880 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.382881 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.382948 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.383329 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:19.383329 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:19.383371 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 02:12:19.383428 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 02:12:19.383529 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 02:12:19.383749 19274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.374596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33102 (local address 127.5.9.189:34737)
0809 02:12:19.374770 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:19.374775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:19.374801 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:12:19.374824 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:19.374826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:19.374841 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:19.374911 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:19.375259 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.375980 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.377028 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.377546 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.377711 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.380083 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:19.380097 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:19.380098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:19.380107 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:19.382312 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.382718 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.382722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.382723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.382803 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.383013 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.383014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.383015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.383257 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:19.383358 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:19.383483 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 02:12:19.383608 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
I20260809 02:12:19.384407 19270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:12:19.384793 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.384669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:43429 (local address 127.0.0.1:34850)
0809 02:12:19.384716 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.384772 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 02:12:19.384989 19216 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.9.188:43429: connect: Connection refused (error 111)
I20260809 02:12:19.385466 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.385377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:43429 (local address 127.0.0.1:34860)
0809 02:12:19.385420 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.385447 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 02:12:19.385648 19270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:43429: Network error: Client connection negotiation failed: client connection to 127.5.9.188:43429: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.188.keytab.
Entry for principal kudu/127.5.9.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.188.keytab.
I20260809 02:12:19.393332 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.188:43429
--webserver_interface=127.5.9.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.188.keytab
--principal=kudu/127.5.9.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:41813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:44559,127.5.9.189:34737,127.5.9.188:43429
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:12:19.411877 19193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } attempt: 3
Aug 09 02:12:19 dist-test-slave-p3t5 krb5kdc[19110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241539, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for kudu/127.5.9.189@KRBTEST.COM
I20260809 02:12:19.419354 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.412300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:34737 (local address 127.0.0.1:33104)
0809 02:12:19.412372 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.412390 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:19.412475 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:19.412651 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:19.412652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:19.412829 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:19.412900 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.412902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.413605 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.413606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.414159 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.414161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.414565 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.414566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.414657 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.415185 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:19.415190 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:19.416633 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:19.418275 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.418277 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.418277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.418466 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.418670 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.418671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.418671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.418737 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.418999 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:19.419000 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:19.419050 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 02:12:19.419132 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 02:12:19.419247 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 02:12:19.419354 19274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.412383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33104 (local address 127.5.9.189:34737)
0809 02:12:19.412456 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:19.412459 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:19.412471 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:12:19.412553 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:19.412554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:19.412577 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:19.412640 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:19.412989 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.413507 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.414253 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.414478 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.414566 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.416721 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:19.416722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:19.416723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:19.416735 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:19.418190 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.418538 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.418540 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.418540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.418603 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.418800 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.418801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.418801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.418924 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:19.418999 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:19.419160 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 02:12:19.419248 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:12:19.419858 19193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:12:19.420260 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.420156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:43429 (local address 127.0.0.1:34874)
0809 02:12:19.420205 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.420237 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 02:12:19.420603 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.420505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:43429 (local address 127.0.0.1:34880)
0809 02:12:19.420546 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.420574 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 02:12:19.420681 19193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:43429: Network error: Client connection negotiation failed: client connection to 127.5.9.188:43429: connect: Connection refused (error 111)
I20260809 02:12:19.435060 19270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } attempt: 1
I20260809 02:12:19.435627 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.435491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:43429 (local address 127.0.0.1:34890)
0809 02:12:19.435551 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.435593 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 02:12:19.436020 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.435921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:43429 (local address 127.0.0.1:34896)
0809 02:12:19.435965 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.435998 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 02:12:19.436110 19270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:43429: Network error: Client connection negotiation failed: client connection to 127.5.9.188:43429: connect: Connection refused (error 111)
I20260809 02:12:19.451840 19193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } attempt: 1
I20260809 02:12:19.452364 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.452229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:43429 (local address 127.0.0.1:34900)
0809 02:12:19.452291 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.452331 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 02:12:19.452720 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.452621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:43429 (local address 127.0.0.1:34906)
0809 02:12:19.452667 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.452694 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 02:12:19.452811 19193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:43429: Network error: Client connection negotiation failed: client connection to 127.5.9.188:43429: connect: Connection refused (error 111)
W20260809 02:12:19.482050 19277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:19.482293 19277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:19.482319 19277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:19.483573 19277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:12:19.483630 19277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:19.483645 19277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:19.483659 19277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:12:19.483708 19277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:12:19.485370 19277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:41813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:44559,127.5.9.189:34737,127.5.9.188:43429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.188.keytab
--principal=kudu/127.5.9.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.188:43429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.9.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:19.485694 19277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:19.485970 19277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:19.491289 19286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:19.491276 19283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:19.491303 19284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:19.491654 19277 server_base.cc:1034] running on GCE node
I20260809 02:12:19.491950 19277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:19.492283 19277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:19.493480 19277 hybrid_clock.cc:648] HybridClock initialized: now 1786241539493440 us; error 57 us; skew 500 ppm
Aug 09 02:12:19 dist-test-slave-p3t5 krb5kdc[19110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241539, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:19.496299 19277 init.cc:377] Logged in from keytab as kudu/127.5.9.188@KRBTEST.COM (short username kudu)
I20260809 02:12:19.496796 19277 webserver.cc:460] Webserver started at http://127.5.9.188:35255/ using document root <none> and password file <none>
I20260809 02:12:19.497001 19277 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:19.497044 19277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:19.497143 19277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:12:19.498340 19277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/data/instance:
uuid: "64231ca67773439288065d7b90aa6143"
format_stamp: "Formatted at 2026-08-09 02:12:19 on dist-test-slave-p3t5"
server_key: "478cdfdadebbb109565238e65ce0790b"
server_key_iv: "ce915604733a024d7f2b1a4c07c03870"
server_key_version: "encryptionkey@0"
I20260809 02:12:19.498673 19277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/wal/instance:
uuid: "64231ca67773439288065d7b90aa6143"
format_stamp: "Formatted at 2026-08-09 02:12:19 on dist-test-slave-p3t5"
server_key: "478cdfdadebbb109565238e65ce0790b"
server_key_iv: "ce915604733a024d7f2b1a4c07c03870"
server_key_version: "encryptionkey@0"
I20260809 02:12:19.500133 19277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:12:19.500939 19293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:19.501147 19277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:19.501269 19277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/wal
uuid: "64231ca67773439288065d7b90aa6143"
format_stamp: "Formatted at 2026-08-09 02:12:19 on dist-test-slave-p3t5"
server_key: "478cdfdadebbb109565238e65ce0790b"
server_key_iv: "ce915604733a024d7f2b1a4c07c03870"
server_key_version: "encryptionkey@0"
I20260809 02:12:19.501356 19277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:19.504266 19270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } attempt: 2
I20260809 02:12:19.504742 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.504606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:43429 (local address 127.0.0.1:34916)
0809 02:12:19.504668 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.504703 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 02:12:19.505138 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.505026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:43429 (local address 127.0.0.1:34920)
0809 02:12:19.505076 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.505113 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:43429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 02:12:19.505235 19270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:43429: Network error: Client connection negotiation failed: client connection to 127.5.9.188:43429: connect: Connection refused (error 111)
I20260809 02:12:19.513432 19277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:19.516024 19277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:19.516198 19277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:19.520947 19277 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.188:43429
I20260809 02:12:19.520984 19345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.188:43429 every 8 connection(s)
I20260809 02:12:19.521384 19277 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/data/info.pb
I20260809 02:12:19.522398 19346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:19.524041 19346 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:12:19.527958 19193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } attempt: 2
I20260809 02:12:19.529733 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 19277
I20260809 02:12:19.529908 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241377393328-5158-0/minicluster-data/master-2/wal/instance
I20260809 02:12:19.530143 5158 external_mini_cluster.cc:1428] Setting key 6da6f5f0f4919b237c7812cc76ca5321
Aug 09 02:12:19 dist-test-slave-p3t5 krb5kdc[19110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241539, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.188@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
Aug 09 02:12:19 dist-test-slave-p3t5 krb5kdc[19110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241539, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for kudu/127.5.9.188@KRBTEST.COM
I20260809 02:12:19.535218 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.524823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50974 (local address 127.5.9.190:44559)
0809 02:12:19.524894 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:19.524896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:19.525136 (+ 240us) server_negotiation.cc:407] Connection header received
0809 02:12:19.525707 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:19.525709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:19.525720 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:19.525793 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:19.526470 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.526979 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.528035 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.528329 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.528413 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.531331 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:19.531333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:19.531334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:19.531343 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:19.533256 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.533829 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.533830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.533831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.533895 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.534190 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.534192 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.534192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.534349 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:19.534422 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:19.534779 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 02:12:19.535131 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:12:19.535236 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.524775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:44559 (local address 127.0.0.1:50974)
0809 02:12:19.524994 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.525045 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:19.525528 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:19.525881 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:19.525885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:19.526164 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:19.526385 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.526389 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.527084 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.527085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.527940 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.527942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.528427 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.528428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.528586 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.529228 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:19.529240 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:19.531230 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:19.533347 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.533357 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.533359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.533746 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.533953 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.533954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.533955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.534094 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.534430 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:19.534438 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:19.534617 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 02:12:19.534738 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 02:12:19.534934 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":121,"threads_started":1}
Aug 09 02:12:19 dist-test-slave-p3t5 krb5kdc[19110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.188@KRBTEST.COM
I20260809 02:12:19.536607 19346 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:12:19.537750 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.528338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:43429 (local address 127.0.0.1:34926)
0809 02:12:19.528402 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.528415 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:19.528457 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:19.528913 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:19.528914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:19.529119 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:19.529163 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.529165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.530491 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.530492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.531255 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.531258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.531798 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.531799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.531864 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.532237 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:19.532242 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:19.533823 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:19.536056 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.536056 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.536057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.536306 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.536536 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.536537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.536537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.536611 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.537362 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:19.537363 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:19.537419 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 02:12:19.537504 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 02:12:19.537645 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 02:12:19.538255 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.528536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34926 (local address 127.5.9.188:43429)
0809 02:12:19.528725 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:19.528729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:19.528782 (+ 53us) server_negotiation.cc:407] Connection header received
0809 02:12:19.528802 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:19.528803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:19.528820 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:19.528884 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:19.529256 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.530386 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.531346 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.531712 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.531784 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.533894 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:19.533901 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:19.533901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:19.533907 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:19.535967 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.536386 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.536388 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.536389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.536458 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.536775 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.536776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.536777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.537282 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:19.537377 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:19.537557 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 02:12:19.538145 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":122,"threads_started":1}
I20260809 02:12:19.540230 19193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a: Bootstrap starting.
Aug 09 02:12:19 dist-test-slave-p3t5 krb5kdc[19110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241539, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.188@KRBTEST.COM for kudu/127.5.9.189@KRBTEST.COM
I20260809 02:12:19.541266 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.530531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34928 (local address 127.5.9.188:43429)
0809 02:12:19.530707 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:19.530708 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:19.530866 (+ 158us) server_negotiation.cc:407] Connection header received
0809 02:12:19.530978 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:19.530979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:19.530991 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:19.531056 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:19.531840 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.532637 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.533445 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.533753 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.533814 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.536675 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:19.536676 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:19.536676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:19.536684 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:19.539629 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.540389 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.540390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.540390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.540463 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.540685 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.540686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.540686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.540805 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:19.540992 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:19.541030 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 02:12:19.541120 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":92,"threads_started":1}
I20260809 02:12:19.541450 19193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:19.541852 19193 log.cc:824] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:19.542896 19193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a: No bootstrap required, opened a new log
I20260809 02:12:19.544927 19193 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } }
I20260809 02:12:19.545102 19193 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:19.545166 19193 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94d13722d55f49dc8c00df6ba37fb14a, State: Initialized, Role: FOLLOWER
I20260809 02:12:19.545306 19274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.536965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33106 (local address 127.5.9.189:34737)
0809 02:12:19.537012 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:19.537014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:19.537023 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:12:19.537070 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:19.537071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:19.537079 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:19.537126 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:19.537577 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.538379 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.539225 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.539448 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.539523 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.542050 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:19.542052 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:19.542053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:19.542063 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:19.543948 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.544381 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.544382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.544383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.544478 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.544703 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.544704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.544704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.544854 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:19.544923 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:19.545111 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 02:12:19.545206 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:12:19.545310 19193 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } }
I20260809 02:12:19.545702 19353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } } }
I20260809 02:12:19.545744 19193 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:12:19.545819 19353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:12:19.545877 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.536904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:34737 (local address 127.0.0.1:33106)
0809 02:12:19.536963 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.536974 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:19.537014 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:19.537154 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:19.537155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:19.537390 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:19.537459 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.537461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.538479 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.538480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.539128 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.539131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.539529 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.539531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.539609 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.540024 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:19.540030 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:19.541898 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:19.544048 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.544049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.544050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.544294 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.544551 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.544552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.544553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.544630 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.544929 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:19.544930 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:19.544984 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 02:12:19.545063 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:12:19.545765 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 02:12:19.546314 19346 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:12:19 dist-test-slave-p3t5 krb5kdc[19110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
W20260809 02:12:19.548619 19364 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:12:19.548682 19364 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 02:12:19 dist-test-slave-p3t5 krb5kdc[19110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241539, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.188@KRBTEST.COM for kudu/127.5.9.188@KRBTEST.COM
I20260809 02:12:19.549248 19123 master_runner.cc:186] Error getting master registration for 127.5.9.189:34737: OK, instance_id { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" instance_seqno: 1786241539351376 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:12:19.551769 19123 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:12:19.554037 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.541894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50990 (local address 127.5.9.190:44559)
0809 02:12:19.541977 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:19.541979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:19.541988 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:12:19.542012 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:19.542012 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:19.542022 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:19.542079 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:19.542730 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.543469 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.544493 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.544783 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.544827 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.548025 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:19.548027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:19.548027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:19.548035 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:19.550089 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.552238 (+ 2149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.552240 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.552240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.552311 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.552513 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.552514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.552514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.552701 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:19.552858 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:19.553838 (+ 980us) server_negotiation.cc:299] Negotiation successful
0809 02:12:19.553933 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 02:12:19.555137 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.546630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:43429 (local address 127.0.0.1:34930)
0809 02:12:19.546709 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.546719 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:19.546769 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:19.546928 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:19.546929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:19.547115 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:19.547166 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.547168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.548106 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.548106 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.548922 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.548925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.549460 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.549461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.549526 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.550066 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:19.550072 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:19.551654 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:19.553688 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.553690 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.553690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.554093 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.554317 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.554318 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.554318 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.554381 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.554824 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:19.554825 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:19.554867 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 02:12:19.554928 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 02:12:19.555035 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 02:12:19.555542 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.546741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34930 (local address 127.5.9.188:43429)
0809 02:12:19.546787 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:19.546789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:19.546818 (+ 29us) server_negotiation.cc:407] Connection header received
0809 02:12:19.546841 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:19.546842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:19.546854 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:19.546908 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:19.547275 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.548021 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.549012 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.549366 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.549445 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.551752 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:19.551753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:19.551753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:19.551761 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:19.553504 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.554177 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.554178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.554179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.554246 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.554496 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.554498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.554498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.554726 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:19.554799 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:19.555367 (+ 568us) server_negotiation.cc:299] Negotiation successful
0809 02:12:19.555457 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 02:12:19.558018 19346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143: Bootstrap starting.
I20260809 02:12:19.558938 19346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:19.559320 19346 log.cc:824] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:19.560194 19346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143: No bootstrap required, opened a new log
Aug 09 02:12:19 dist-test-slave-p3t5 krb5kdc[19110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.189@KRBTEST.COM
I20260809 02:12:19.562502 19346 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } }
I20260809 02:12:19.562688 19346 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:19.562729 19346 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64231ca67773439288065d7b90aa6143, State: Initialized, Role: FOLLOWER
I20260809 02:12:19.562870 19346 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } }
I20260809 02:12:19.563180 19367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } } }
I20260809 02:12:19.563272 19346 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:12:19.563338 19367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:12:19.564812 19274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.554797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33116 (local address 127.5.9.189:34737)
0809 02:12:19.554856 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:19.554858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:19.554899 (+ 41us) server_negotiation.cc:407] Connection header received
0809 02:12:19.554976 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:19.554977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:19.554988 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:19.555044 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:19.555621 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.556226 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.557195 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.557478 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.557525 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.560842 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:19.560843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:19.560844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:19.560854 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:19.562856 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.563305 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.563307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.563307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.563379 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.563605 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.563606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.563607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.563785 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:19.564024 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:19.564560 (+ 536us) server_negotiation.cc:299] Negotiation successful
0809 02:12:19.564672 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 02:12:19.566807 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 02:12:19.567003 19378 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:12:19.567132 19378 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:12:19.568331 19277 master_runner.cc:186] Error getting master registration for 127.5.9.189:34737: OK, instance_id { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" instance_seqno: 1786241539351376 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:12:19.571130 19277 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:12:19.577128 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.568259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50998 (local address 127.5.9.190:44559)
0809 02:12:19.568326 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:19.568328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:19.568880 (+ 552us) server_negotiation.cc:407] Connection header received
0809 02:12:19.568976 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:19.568977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:19.568989 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:19.569020 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:19.569952 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.570675 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.571892 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.572165 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.572210 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.574011 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:19.574013 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:19.574014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:19.574022 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:19.575769 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.576197 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.576198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.576199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.576263 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.576473 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.576474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.576474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.576617 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:19.576695 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:19.576935 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 02:12:19.577022 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:12:19.578375 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.569695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34940 (local address 127.5.9.188:43429)
0809 02:12:19.569745 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:19.569747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:19.569756 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:12:19.569778 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:19.569779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:19.569789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:19.569850 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:19.570553 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.571335 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.572383 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.573030 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.573096 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.575149 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:19.575150 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:19.575151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:19.575159 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:19.576816 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.577203 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.577204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.577204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.577260 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.577476 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.577477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.577477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.577603 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:19.577729 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:19.578174 (+ 445us) server_negotiation.cc:299] Negotiation successful
0809 02:12:19.578262 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:12:19.586687 19274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.568607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33118 (local address 127.5.9.189:34737)
0809 02:12:19.568646 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:19.568647 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:19.568943 (+ 296us) server_negotiation.cc:407] Connection header received
0809 02:12:19.569041 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:19.569042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:19.569052 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:19.569083 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:19.579015 (+ 9932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.579834 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.580867 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.581176 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.581257 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.582984 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:19.582986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:19.582987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:19.582998 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:19.584803 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.585253 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.585254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.585255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.585326 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.585553 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.585554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.585554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.585759 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:19.586440 (+ 681us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:19.586486 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 02:12:19.586577 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 02:12:19.596400 19270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } attempt: 3
Aug 09 02:12:19 dist-test-slave-p3t5 krb5kdc[19110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241539, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.189@KRBTEST.COM for kudu/127.5.9.188@KRBTEST.COM
I20260809 02:12:19.605677 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.596803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:43429 (local address 127.0.0.1:34942)
0809 02:12:19.596871 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:19.596884 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:19.596927 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:19.597122 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:19.597123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:19.597349 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:19.597411 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.597413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.598462 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.598464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.599351 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:19.599355 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.599986 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.599987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:19.600058 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.600647 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:19.600654 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:19.602359 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:19.604407 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.604408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.604409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.604665 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.604918 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:19.604919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:19.604919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:19.604989 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:19.605301 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:19.605302 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:19.605353 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 02:12:19.605405 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 02:12:19.605524 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 02:12:19.606293 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:19.596908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34942 (local address 127.5.9.188:43429)
0809 02:12:19.596973 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:19.596975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:19.596986 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:12:19.597017 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:19.597018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:19.597029 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:19.597105 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:19.597502 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.598337 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.599465 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:19.599815 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:19.599896 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:19.602450 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:19.602452 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:19.602452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:19.602462 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:19.604317 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.604756 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.604758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.604758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.604828 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:19.605057 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:19.605058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:19.605059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:19.605201 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:19.606067 (+ 866us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:19.606114 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 02:12:19.606202 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:12:19.608319 19270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715: Bootstrap starting.
I20260809 02:12:19.609148 19270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:19.609500 19270 log.cc:824] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:19.610577 19270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715: No bootstrap required, opened a new log
I20260809 02:12:19.612730 19270 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } }
I20260809 02:12:19.612895 19270 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:19.612932 19270 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90cd4b92c72e46418aa06483c5ae3715, State: Initialized, Role: FOLLOWER
I20260809 02:12:19.613096 19270 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } }
I20260809 02:12:19.613469 19270 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:12:19.613427 19389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } } }
I20260809 02:12:19.613652 19389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 02:12:19.616287 19400 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:12:19.616410 19400 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:12:19.617827 19200 master_runner.cc:263] No leader master found from master 90cd4b92c72e46418aa06483c5ae3715
I20260809 02:12:19.617878 19200 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 02:12:19.681730 19145 tablet.cc:2369] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:12:19.754522 19389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:12:19.754678 19389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } }
I20260809 02:12:19.754973 19389 leader_election.cc:288] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94d13722d55f49dc8c00df6ba37fb14a (127.5.9.190:44559), 64231ca67773439288065d7b90aa6143 (127.5.9.188:43429)
I20260809 02:12:19.755313 19166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90cd4b92c72e46418aa06483c5ae3715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94d13722d55f49dc8c00df6ba37fb14a" is_pre_election: true
I20260809 02:12:19.755314 19320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90cd4b92c72e46418aa06483c5ae3715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64231ca67773439288065d7b90aa6143" is_pre_election: true
I20260809 02:12:19.755503 19166 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90cd4b92c72e46418aa06483c5ae3715 in term 0.
I20260809 02:12:19.755505 19320 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90cd4b92c72e46418aa06483c5ae3715 in term 0.
I20260809 02:12:19.755738 19216 leader_election.cc:302] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64231ca67773439288065d7b90aa6143, 90cd4b92c72e46418aa06483c5ae3715; no voters:
I20260809 02:12:19.755920 19389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:12:19.756008 19389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:12:19.756032 19389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:19.757022 19389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } }
I20260809 02:12:19.757189 19389 leader_election.cc:288] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [CANDIDATE]: Term 1 election: Requested vote from peers 94d13722d55f49dc8c00df6ba37fb14a (127.5.9.190:44559), 64231ca67773439288065d7b90aa6143 (127.5.9.188:43429)
I20260809 02:12:19.757419 19320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90cd4b92c72e46418aa06483c5ae3715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64231ca67773439288065d7b90aa6143"
I20260809 02:12:19.757428 19166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90cd4b92c72e46418aa06483c5ae3715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94d13722d55f49dc8c00df6ba37fb14a"
I20260809 02:12:19.757525 19320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:19.757541 19166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:19.758507 19166 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90cd4b92c72e46418aa06483c5ae3715 in term 1.
I20260809 02:12:19.758684 19320 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90cd4b92c72e46418aa06483c5ae3715 in term 1.
I20260809 02:12:19.758737 19219 leader_election.cc:302] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90cd4b92c72e46418aa06483c5ae3715, 94d13722d55f49dc8c00df6ba37fb14a; no voters:
I20260809 02:12:19.758985 19389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:12:19.759191 19389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [term 1 LEADER]: Becoming Leader. State: Replica: 90cd4b92c72e46418aa06483c5ae3715, State: Running, Role: LEADER
I20260809 02:12:19.759318 19389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } }
I20260809 02:12:19.759829 19402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90cd4b92c72e46418aa06483c5ae3715. Latest consensus state: current_term: 1 leader_uuid: "90cd4b92c72e46418aa06483c5ae3715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } } }
I20260809 02:12:19.759934 19402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [sys.catalog]: This master's current role is: LEADER
I20260809 02:12:19.760169 19404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:12:19.760411 19404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:12:19.762089 19166 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a [term 1 FOLLOWER]: Refusing update from remote peer 90cd4b92c72e46418aa06483c5ae3715: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:12:19.762097 19320 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143 [term 1 FOLLOWER]: Refusing update from remote peer 90cd4b92c72e46418aa06483c5ae3715: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:12:19.762486 19402 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:12:19.762614 19389 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:12:19.764003 19353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90cd4b92c72e46418aa06483c5ae3715. Latest consensus state: current_term: 1 leader_uuid: "90cd4b92c72e46418aa06483c5ae3715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } } }
I20260809 02:12:19.764109 19353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:12:19.764297 19367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90cd4b92c72e46418aa06483c5ae3715. Latest consensus state: current_term: 1 leader_uuid: "90cd4b92c72e46418aa06483c5ae3715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } } }
I20260809 02:12:19.764365 19367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 02:12:19.766871 19299 tablet.cc:2369] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:12:19.767309 19405 mvcc.cc:205] Tried to move back new op lower bound from 7316445346862465024 to 7316445346854768640. Current Snapshot: MvccSnapshot[applied={T|T < 7316445346862465024}]
I20260809 02:12:19.768006 19353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "90cd4b92c72e46418aa06483c5ae3715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } } }
I20260809 02:12:19.768023 19367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "90cd4b92c72e46418aa06483c5ae3715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } } }
I20260809 02:12:19.768107 19353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:12:19.768118 19367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:12:19.768530 19404 catalog_manager.cc:1347] Generated new cluster ID: ae3c74fd96ae4b288773159fe8db409f
I20260809 02:12:19.768595 19404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:12:19.768556 19411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "90cd4b92c72e46418aa06483c5ae3715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } } }
I20260809 02:12:19.768870 19411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [sys.catalog]: This master's current role is: LEADER
I20260809 02:12:19.768659 19402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "90cd4b92c72e46418aa06483c5ae3715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d13722d55f49dc8c00df6ba37fb14a" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 44559 } } peers { permanent_uuid: "90cd4b92c72e46418aa06483c5ae3715" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 34737 } } peers { permanent_uuid: "64231ca67773439288065d7b90aa6143" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 43429 } } }
I20260809 02:12:19.769783 19402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715 [sys.catalog]: This master's current role is: LEADER
I20260809 02:12:19.781410 19404 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:12:19.782192 19404 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:12:19.789042 19404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90cd4b92c72e46418aa06483c5ae3715: Generated new TSK 0
I20260809 02:12:19.789286 19404 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:12:20.549144 19364 catalog_manager.cc:1259] Loaded cluster ID: ae3c74fd96ae4b288773159fe8db409f
I20260809 02:12:20.549235 19364 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a: loading cluster ID for follower catalog manager: success
I20260809 02:12:20.550122 19364 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a: acquiring CA information for follower catalog manager: success
I20260809 02:12:20.550457 19364 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 94d13722d55f49dc8c00df6ba37fb14a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:12:20.567613 19378 catalog_manager.cc:1259] Loaded cluster ID: ae3c74fd96ae4b288773159fe8db409f
I20260809 02:12:20.567703 19378 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143: loading cluster ID for follower catalog manager: success
I20260809 02:12:20.568694 19378 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143: acquiring CA information for follower catalog manager: success
I20260809 02:12:20.568984 19378 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 64231ca67773439288065d7b90aa6143: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:12:20.857653 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 19123
I20260809 02:12:20.864833 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 19200
I20260809 02:12:20.872220 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 19277
2026-08-09T02:12:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3023 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:12:20 dist-test-slave-p3t5 krb5kdc[19421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:12:20 dist-test-slave-p3t5 krb5kdc[19421](info): set up 2 sockets
Aug 09 02:12:20 dist-test-slave-p3t5 krb5kdc[19421](info): commencing operation
krb5kdc: starting...
W20260809 02:12:21.946587 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:12:22 dist-test-slave-p3t5 krb5kdc[19421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:12:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:12:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:12:22.080832 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:39307
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:42705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:39307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:22.167613 19435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:22.167852 19435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:22.167877 19435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:22.169121 19435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:12:22.169176 19435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:22.169191 19435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:22.169204 19435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:12:22.169261 19435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:12:22.170841 19435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:42705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:39307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:39307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:22.171128 19435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:22.171387 19435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:22.176651 19440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:22.176663 19443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:22.176790 19435 server_base.cc:1034] running on GCE node
W20260809 02:12:22.176676 19441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:22.177311 19435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:22.177711 19435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:22.178881 19435 hybrid_clock.cc:648] HybridClock initialized: now 1786241542178860 us; error 49 us; skew 500 ppm
Aug 09 02:12:22 dist-test-slave-p3t5 krb5kdc[19421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241542, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:22.187273 19435 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:12:22.187798 19435 webserver.cc:460] Webserver started at http://127.5.9.190:41039/ using document root <none> and password file <none>
I20260809 02:12:22.188074 19435 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:22.188146 19435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:22.188310 19435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:12:22.189476 19435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "38f6140201aa4e97a71589c58ca161b8"
format_stamp: "Formatted at 2026-08-09 02:12:22 on dist-test-slave-p3t5"
server_key: "b5de5846cd2cb3ddeeaa52c0512f747e"
server_key_iv: "2d478c8e786600d74b4da98bde9a624d"
server_key_version: "encryptionkey@0"
I20260809 02:12:22.189970 19435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "38f6140201aa4e97a71589c58ca161b8"
format_stamp: "Formatted at 2026-08-09 02:12:22 on dist-test-slave-p3t5"
server_key: "b5de5846cd2cb3ddeeaa52c0512f747e"
server_key_iv: "2d478c8e786600d74b4da98bde9a624d"
server_key_version: "encryptionkey@0"
I20260809 02:12:22.191550 19435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:12:22.192488 19450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:22.192747 19435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:22.192864 19435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "38f6140201aa4e97a71589c58ca161b8"
format_stamp: "Formatted at 2026-08-09 02:12:22 on dist-test-slave-p3t5"
server_key: "b5de5846cd2cb3ddeeaa52c0512f747e"
server_key_iv: "2d478c8e786600d74b4da98bde9a624d"
server_key_version: "encryptionkey@0"
I20260809 02:12:22.192961 19435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:22.208189 19435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:22.210835 19435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:22.211028 19435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:22.215613 19435 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:39307
I20260809 02:12:22.215682 19504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:39307 every 8 connection(s)
I20260809 02:12:22.216081 19435 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:12:22.217075 19505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:22.217134 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 19435
I20260809 02:12:22.217244 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:12:22.217474 5158 external_mini_cluster.cc:1428] Setting key 9ff4726ce70699f7c48078ea7b055e54
I20260809 02:12:22.220813 19505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8: Bootstrap starting.
I20260809 02:12:22.221591 19505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:22.221961 19505 log.cc:824] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:22.222721 19505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8: No bootstrap required, opened a new log
Aug 09 02:12:22 dist-test-slave-p3t5 krb5kdc[19421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:22.224702 19505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f6140201aa4e97a71589c58ca161b8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39307 } }
I20260809 02:12:22.224889 19505 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:22.224939 19505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38f6140201aa4e97a71589c58ca161b8, State: Initialized, Role: FOLLOWER
I20260809 02:12:22.225109 19505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f6140201aa4e97a71589c58ca161b8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39307 } }
I20260809 02:12:22.225205 19505 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:12:22.225251 19505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:12:22.225301 19505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:22.226150 19505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f6140201aa4e97a71589c58ca161b8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39307 } }
I20260809 02:12:22.226277 19505 leader_election.cc:302] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38f6140201aa4e97a71589c58ca161b8; no voters:
I20260809 02:12:22.226526 19505 leader_election.cc:288] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:12:22.226563 19510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:12:22.226684 19510 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [term 1 LEADER]: Becoming Leader. State: Replica: 38f6140201aa4e97a71589c58ca161b8, State: Running, Role: LEADER
I20260809 02:12:22.226784 19510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f6140201aa4e97a71589c58ca161b8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39307 } }
I20260809 02:12:22.226848 19505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:12:22.227141 19510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38f6140201aa4e97a71589c58ca161b8. Latest consensus state: current_term: 1 leader_uuid: "38f6140201aa4e97a71589c58ca161b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f6140201aa4e97a71589c58ca161b8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39307 } } }
I20260809 02:12:22.227217 19510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [sys.catalog]: This master's current role is: LEADER
I20260809 02:12:22.227176 19511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38f6140201aa4e97a71589c58ca161b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f6140201aa4e97a71589c58ca161b8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 39307 } } }
I20260809 02:12:22.227411 19511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8 [sys.catalog]: This master's current role is: LEADER
W20260809 02:12:22.228855 19526 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:12:22.228981 19526 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:12:22.229111 19515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:12:22.229310 19515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:12:22.230122 19508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:22.218087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52512 (local address 127.5.9.190:39307)
0809 02:12:22.218283 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:22.218294 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:22.218331 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:12:22.218859 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:22.218869 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:22.218950 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:22.219115 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:22.219743 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:22.221218 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:22.222098 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:22.222522 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:22.222644 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:22.225152 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:22.225157 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:22.225157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:22.225169 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:22.227418 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:22.227993 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:22.227997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:22.227997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:22.228124 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:22.228426 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:22.228427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:22.228428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:22.228776 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:22.229678 (+ 902us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:22.229757 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 02:12:22.229914 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260809 02:12:22.230955 19515 catalog_manager.cc:1347] Generated new cluster ID: 273f457250ba49a2b462b496912c66b5
I20260809 02:12:22.231010 19515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:12:22.231042 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:12:22.250092 19515 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:12:22.250895 19515 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:12:22.254746 19515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38f6140201aa4e97a71589c58ca161b8: Generated new TSK 0
I20260809 02:12:22.254974 19515 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:12:22.285591 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 19435
2026-08-09T02:12:22Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1412 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:12:22 dist-test-slave-p3t5 krb5kdc[19533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:12:22 dist-test-slave-p3t5 krb5kdc[19533](info): set up 2 sockets
Aug 09 02:12:22 dist-test-slave-p3t5 krb5kdc[19533](info): commencing operation
krb5kdc: starting...
W20260809 02:12:23.378620 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:12:23 dist-test-slave-p3t5 krb5kdc[19533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:12:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:12:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:12:23.517382 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:45531
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:45531
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:23.604089 19548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:23.604398 19548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:23.604441 19548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:23.605718 19548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:12:23.605800 19548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:23.605827 19548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:23.605870 19548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:12:23.605893 19548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:12:23.607695 19548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:45531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:45531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:23.608021 19548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:23.608275 19548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:23.613677 19553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:23.613677 19557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:23.613720 19555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:23.613706 19548 server_base.cc:1034] running on GCE node
I20260809 02:12:23.614285 19548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:23.614651 19548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:23.615837 19548 hybrid_clock.cc:648] HybridClock initialized: now 1786241543615817 us; error 44 us; skew 500 ppm
Aug 09 02:12:23 dist-test-slave-p3t5 krb5kdc[19533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241543, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:23.618779 19548 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:12:23.619393 19548 webserver.cc:460] Webserver started at http://127.5.9.190:40169/ using document root <none> and password file <none>
I20260809 02:12:23.619663 19548 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:23.619735 19548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:23.619894 19548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:12:23.621099 19548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "c34349d7c2bc42d5b6d3c548d6521f05"
format_stamp: "Formatted at 2026-08-09 02:12:23 on dist-test-slave-p3t5"
server_key: "92f314b4514248be5f67bb2fafd23d5b"
server_key_iv: "ea65473e18079b7b046e9b3027c0cfbe"
server_key_version: "encryptionkey@0"
I20260809 02:12:23.621496 19548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "c34349d7c2bc42d5b6d3c548d6521f05"
format_stamp: "Formatted at 2026-08-09 02:12:23 on dist-test-slave-p3t5"
server_key: "92f314b4514248be5f67bb2fafd23d5b"
server_key_iv: "ea65473e18079b7b046e9b3027c0cfbe"
server_key_version: "encryptionkey@0"
I20260809 02:12:23.623009 19548 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:23.623880 19564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:23.624061 19548 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:12:23.624171 19548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "c34349d7c2bc42d5b6d3c548d6521f05"
format_stamp: "Formatted at 2026-08-09 02:12:23 on dist-test-slave-p3t5"
server_key: "92f314b4514248be5f67bb2fafd23d5b"
server_key_iv: "ea65473e18079b7b046e9b3027c0cfbe"
server_key_version: "encryptionkey@0"
I20260809 02:12:23.624251 19548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:23.653218 19548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:23.655820 19548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:23.656002 19548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:23.660429 19548 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:45531
I20260809 02:12:23.660487 19616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:45531 every 8 connection(s)
I20260809 02:12:23.660876 19548 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:12:23.661998 19617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:23.664089 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 19548
I20260809 02:12:23.664201 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:12:23.664424 5158 external_mini_cluster.cc:1428] Setting key b8d93e9e7b686294754d910585f81771
I20260809 02:12:23.664851 19617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: Bootstrap starting.
I20260809 02:12:23.665688 19617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:23.666009 19617 log.cc:824] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:23.666798 19617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: No bootstrap required, opened a new log
I20260809 02:12:23.668661 19617 raft_consensus.cc:348] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } }
I20260809 02:12:23.668834 19617 raft_consensus.cc:374] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:23.668887 19617 raft_consensus.cc:729] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c34349d7c2bc42d5b6d3c548d6521f05, State: Initialized, Role: FOLLOWER
I20260809 02:12:23.669060 19617 consensus_queue.cc:260] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } }
I20260809 02:12:23.669167 19617 raft_consensus.cc:388] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:12:23.669231 19617 raft_consensus.cc:482] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:12:23.669286 19617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:23.669983 19617 raft_consensus.cc:504] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } }
I20260809 02:12:23.670112 19617 leader_election.cc:302] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c34349d7c2bc42d5b6d3c548d6521f05; no voters:
I20260809 02:12:23.670318 19617 leader_election.cc:288] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:12:23.670344 19623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:12:23.670500 19623 raft_consensus.cc:686] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 1 LEADER]: Becoming Leader. State: Replica: c34349d7c2bc42d5b6d3c548d6521f05, State: Running, Role: LEADER
I20260809 02:12:23.670655 19617 sys_catalog.cc:564] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:12:23.670675 19623 consensus_queue.cc:237] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } }
I20260809 02:12:23.671133 19624 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c34349d7c2bc42d5b6d3c548d6521f05. Latest consensus state: current_term: 1 leader_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } } }
I20260809 02:12:23.671231 19624 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [sys.catalog]: This master's current role is: LEADER
Aug 09 02:12:23 dist-test-slave-p3t5 krb5kdc[19533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:23.671502 19625 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } } }
I20260809 02:12:23.671993 19625 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [sys.catalog]: This master's current role is: LEADER
I20260809 02:12:23.672183 19630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:12:23.672657 19630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:12:23.677809 19621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:23.665046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33142 (local address 127.5.9.190:45531)
0809 02:12:23.665235 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:23.665243 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:23.665271 (+ 28us) server_negotiation.cc:407] Connection header received
0809 02:12:23.665912 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:23.665920 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:23.665975 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:23.666136 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:23.666808 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:23.667746 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:23.668498 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:23.668984 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:23.669107 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:23.673390 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:23.673395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:23.673395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:23.673409 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:23.675897 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:23.676389 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:23.676393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:23.676393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:23.676515 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:23.676778 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:23.676779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:23.676779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:23.677133 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:23.677298 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:23.677376 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 02:12:23.677538 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260809 02:12:23.678454 19630 catalog_manager.cc:1347] Generated new cluster ID: 3f04f1a79884492ca17b13b0718aeea3
I20260809 02:12:23.678522 19630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:12:23.687470 19630 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:12:23.688254 19630 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:12:23.701958 19630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: Generated new TSK 0
I20260809 02:12:23.702235 19630 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:12:23.733059 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:45531
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:23.821130 19646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:23.821370 19646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:23.821398 19646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:23.822670 19646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:23.822723 19646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:23.822758 19646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:12:23.824236 19646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:45531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:23.824456 19646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:23.824654 19646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:23.830134 19652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:23.830142 19654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:23.830142 19651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:23.830211 19646 server_base.cc:1034] running on GCE node
I20260809 02:12:23.830574 19646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:23.830791 19646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:23.831957 19646 hybrid_clock.cc:648] HybridClock initialized: now 1786241543831937 us; error 36 us; skew 500 ppm
Aug 09 02:12:23 dist-test-slave-p3t5 krb5kdc[19533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241543, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:23.834713 19646 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:12:23.835197 19646 webserver.cc:460] Webserver started at http://127.5.9.129:40259/ using document root <none> and password file <none>
I20260809 02:12:23.835419 19646 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:23.835462 19646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:23.835557 19646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:12:23.836620 19646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "7d53e1c10ec94ea3b47bcd94c414bb97"
format_stamp: "Formatted at 2026-08-09 02:12:23 on dist-test-slave-p3t5"
server_key: "49a225c8f0824f3017f1f87fffd8646e"
server_key_iv: "bcf460ef7907bf90aa83774f231494d9"
server_key_version: "encryptionkey@0"
I20260809 02:12:23.836943 19646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "7d53e1c10ec94ea3b47bcd94c414bb97"
format_stamp: "Formatted at 2026-08-09 02:12:23 on dist-test-slave-p3t5"
server_key: "49a225c8f0824f3017f1f87fffd8646e"
server_key_iv: "bcf460ef7907bf90aa83774f231494d9"
server_key_version: "encryptionkey@0"
I20260809 02:12:23.838369 19646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:12:23.839103 19661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:23.839277 19646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:12:23.839385 19646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "7d53e1c10ec94ea3b47bcd94c414bb97"
format_stamp: "Formatted at 2026-08-09 02:12:23 on dist-test-slave-p3t5"
server_key: "49a225c8f0824f3017f1f87fffd8646e"
server_key_iv: "bcf460ef7907bf90aa83774f231494d9"
server_key_version: "encryptionkey@0"
I20260809 02:12:23.839468 19646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:23.868921 19646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:23.871414 19646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:23.871598 19646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:23.871945 19646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:23.872292 19646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:12:23.872346 19646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:23.872383 19646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:12:23.872427 19646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:23.878582 19646 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:41507
I20260809 02:12:23.878628 19780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:41507 every 8 connection(s)
I20260809 02:12:23.878988 19646 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:12:23.879490 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 19646
I20260809 02:12:23.879596 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:12:23.879820 5158 external_mini_cluster.cc:1428] Setting key 63880fe2daa8651a3ddbd255d5f24e44
Aug 09 02:12:23 dist-test-slave-p3t5 krb5kdc[19533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241543, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
I20260809 02:12:23.890725 19621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:23.880135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:60029 (local address 127.5.9.190:45531)
0809 02:12:23.880190 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:23.880192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:23.880202 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:12:23.880801 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:23.880802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:23.880812 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:23.880846 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:23.882277 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:23.882804 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:23.884305 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:23.884515 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:23.884608 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:23.887401 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:23.887408 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:23.887409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:23.887417 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:23.889063 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:23.889552 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:23.889553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:23.889554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:23.889663 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:23.889955 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:23.889956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:23.889957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:23.890093 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:23.890169 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:23.890516 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 02:12:23.890623 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 02:12:23.890936 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:23.879777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:60029)
0809 02:12:23.880008 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:23.880060 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:23.880683 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:23.880930 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:23.880934 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:23.881865 (+ 931us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:23.882171 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:23.882178 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:23.882910 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:23.882911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:23.884209 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:23.884212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:23.884624 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:23.884625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:23.884815 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:23.885402 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:23.885411 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:23.887266 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:23.889184 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:23.889195 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:23.889198 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:23.889467 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:23.889738 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:23.889739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:23.889740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:23.889870 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:23.890160 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:23.890168 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:23.890351 (+ 183us) client_negotiation.cc:769] Sending connection context
0809 02:12:23.890478 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 02:12:23.890660 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":111,"threads_started":1}
I20260809 02:12:23.891505 19781 heartbeater.cc:347] Connected to a master server at 127.5.9.190:45531
I20260809 02:12:23.891626 19781 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:23.891829 19781 heartbeater.cc:511] Master 127.5.9.190:45531 requested a full tablet report, sending...
I20260809 02:12:23.892395 19581 ts_manager.cc:194] Registered new tserver with Master: 7d53e1c10ec94ea3b47bcd94c414bb97 (127.5.9.129:41507)
I20260809 02:12:23.893136 19581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:60029
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:12:23.910663 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:45531
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:24.000464 19786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:24.000722 19786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:24.000762 19786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:24.001945 19786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:24.002012 19786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:24.002058 19786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:12:24.003537 19786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:45531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:24.003811 19786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:24.004055 19786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:24.009215 19791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:24.009215 19794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:24.009316 19786 server_base.cc:1034] running on GCE node
W20260809 02:12:24.009234 19792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:24.009747 19786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:24.010082 19786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:24.011242 19786 hybrid_clock.cc:648] HybridClock initialized: now 1786241544011228 us; error 31 us; skew 500 ppm
Aug 09 02:12:24 dist-test-slave-p3t5 krb5kdc[19533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241544, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:24.014127 19786 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:12:24.014647 19786 webserver.cc:460] Webserver started at http://127.5.9.130:43817/ using document root <none> and password file <none>
I20260809 02:12:24.014894 19786 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:24.014964 19786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:24.015118 19786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:12:24.016235 19786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "3c3e4281cf15481197e7667321267601"
format_stamp: "Formatted at 2026-08-09 02:12:24 on dist-test-slave-p3t5"
server_key: "79b0bc1a2234816f701131b3cc3f5cd4"
server_key_iv: "e7b1ff2d86f8cbd7a08923f9a03c7375"
server_key_version: "encryptionkey@0"
I20260809 02:12:24.016674 19786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "3c3e4281cf15481197e7667321267601"
format_stamp: "Formatted at 2026-08-09 02:12:24 on dist-test-slave-p3t5"
server_key: "79b0bc1a2234816f701131b3cc3f5cd4"
server_key_iv: "e7b1ff2d86f8cbd7a08923f9a03c7375"
server_key_version: "encryptionkey@0"
I20260809 02:12:24.018241 19786 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:12:24.019085 19801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:24.019290 19786 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:12:24.019407 19786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "3c3e4281cf15481197e7667321267601"
format_stamp: "Formatted at 2026-08-09 02:12:24 on dist-test-slave-p3t5"
server_key: "79b0bc1a2234816f701131b3cc3f5cd4"
server_key_iv: "e7b1ff2d86f8cbd7a08923f9a03c7375"
server_key_version: "encryptionkey@0"
I20260809 02:12:24.019486 19786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:24.033784 19786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:24.035986 19786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:24.036161 19786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:24.036428 19786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:24.036751 19786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:12:24.036804 19786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:24.036841 19786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:12:24.036864 19786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:24.043309 19786 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:33231
I20260809 02:12:24.043376 19918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:33231 every 8 connection(s)
I20260809 02:12:24.043818 19786 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:12:24.047188 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 19786
I20260809 02:12:24.047327 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:12:24.047537 5158 external_mini_cluster.cc:1428] Setting key 539a9630081eab455a3b1b99e61576fe
Aug 09 02:12:24 dist-test-slave-p3t5 krb5kdc[19533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241544, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:24.055235 19621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.044648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:48091 (local address 127.5.9.190:45531)
0809 02:12:24.044718 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:24.044733 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:24.044939 (+ 206us) server_negotiation.cc:407] Connection header received
0809 02:12:24.045531 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:24.045532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:24.045546 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:24.045605 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:24.046707 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.047545 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.048766 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.049143 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.049220 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.051970 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:24.051972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:24.051973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:24.051981 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:24.053589 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:24.054124 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:24.054125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:24.054126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:24.054199 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:24.054494 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:24.054496 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:24.054496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:24.054635 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:24.054715 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:24.055027 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 02:12:24.055124 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:12:24.055481 19921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.044611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:48091)
0809 02:12:24.044804 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:24.044850 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:24.045410 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:24.045708 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:24.045712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:24.046372 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:24.046588 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.046593 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.047678 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.047680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.048680 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.048683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.049239 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.049239 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.049343 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.050005 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:24.050015 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:24.051860 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:24.053736 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:24.053744 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:24.053747 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:24.054034 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:24.054276 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:24.054277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:24.054277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:24.054414 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:24.054706 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:24.054714 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:24.054880 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 02:12:24.055001 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 02:12:24.055195 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":94,"threads_started":1}
I20260809 02:12:24.056027 19919 heartbeater.cc:347] Connected to a master server at 127.5.9.190:45531
I20260809 02:12:24.056139 19919 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:24.056352 19919 heartbeater.cc:511] Master 127.5.9.190:45531 requested a full tablet report, sending...
I20260809 02:12:24.056810 19581 ts_manager.cc:194] Registered new tserver with Master: 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231)
I20260809 02:12:24.057246 19581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:48091
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:12:24.076651 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:45531
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:24.162840 19925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:24.163111 19925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:24.163153 19925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:24.164381 19925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:24.164459 19925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:24.164508 19925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:12:24.166148 19925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:45531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:24.166433 19925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:24.166683 19925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:24.172083 19933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:24.172083 19931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:24.172118 19930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:24.172365 19925 server_base.cc:1034] running on GCE node
I20260809 02:12:24.172576 19925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:24.172829 19925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:24.173998 19925 hybrid_clock.cc:648] HybridClock initialized: now 1786241544173973 us; error 42 us; skew 500 ppm
Aug 09 02:12:24 dist-test-slave-p3t5 krb5kdc[19533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241544, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:24.176723 19925 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:12:24.177238 19925 webserver.cc:460] Webserver started at http://127.5.9.131:42065/ using document root <none> and password file <none>
I20260809 02:12:24.177496 19925 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:24.177564 19925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:24.177830 19925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:12:24.178972 19925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "92ef8f93814a4ec19caf2e766e221413"
format_stamp: "Formatted at 2026-08-09 02:12:24 on dist-test-slave-p3t5"
server_key: "620790ddd217977932153341cafb26bf"
server_key_iv: "0f7bc02ae5d03fe1da31b6e4e10a7c83"
server_key_version: "encryptionkey@0"
I20260809 02:12:24.179317 19925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "92ef8f93814a4ec19caf2e766e221413"
format_stamp: "Formatted at 2026-08-09 02:12:24 on dist-test-slave-p3t5"
server_key: "620790ddd217977932153341cafb26bf"
server_key_iv: "0f7bc02ae5d03fe1da31b6e4e10a7c83"
server_key_version: "encryptionkey@0"
I20260809 02:12:24.180662 19925 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:12:24.181376 19940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:24.181566 19925 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:12:24.181684 19925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "92ef8f93814a4ec19caf2e766e221413"
format_stamp: "Formatted at 2026-08-09 02:12:24 on dist-test-slave-p3t5"
server_key: "620790ddd217977932153341cafb26bf"
server_key_iv: "0f7bc02ae5d03fe1da31b6e4e10a7c83"
server_key_version: "encryptionkey@0"
I20260809 02:12:24.181759 19925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:24.191895 19925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:24.194396 19925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:24.194571 19925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:24.194842 19925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:24.195170 19925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:12:24.195206 19925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:24.195267 19925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:12:24.195297 19925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:24.201785 19925 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:46001
I20260809 02:12:24.201823 20056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:46001 every 8 connection(s)
I20260809 02:12:24.202257 19925 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:12:24.202838 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 19925
I20260809 02:12:24.202941 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:12:24.203159 5158 external_mini_cluster.cc:1428] Setting key 482dbaf7f83dbd53183f196be0d10c95
Aug 09 02:12:24 dist-test-slave-p3t5 krb5kdc[19533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241544, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:24.212612 19621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.203436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:33375 (local address 127.5.9.190:45531)
0809 02:12:24.203484 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:24.203486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:24.203496 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:12:24.204242 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:24.204243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:24.204255 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:24.204310 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:24.205329 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.205935 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.207012 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.207213 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.207283 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.209606 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:24.209607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:24.209608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:24.209648 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:24.211067 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:24.211550 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:24.211551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:24.211552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:24.211618 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:24.211907 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:24.211908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:24.211909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:24.212043 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:24.212124 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:24.212416 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 02:12:24.212508 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:12:24.212833 20059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.203112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:33375)
0809 02:12:24.203336 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:24.203393 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:24.204127 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:24.204376 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:24.204381 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:24.204999 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:24.205205 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.205210 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.206040 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.206041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.206907 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.206910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.207304 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.207305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.207443 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.208003 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:24.208012 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:24.209498 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:24.211186 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:24.211195 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:24.211197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:24.211468 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:24.211700 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:24.211700 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:24.211700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:24.211828 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:24.212114 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:24.212121 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:24.212277 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 02:12:24.212390 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 02:12:24.212579 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":105,"threads_started":1}
I20260809 02:12:24.213387 20057 heartbeater.cc:347] Connected to a master server at 127.5.9.190:45531
I20260809 02:12:24.213503 20057 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:24.213737 20057 heartbeater.cc:511] Master 127.5.9.190:45531 requested a full tablet report, sending...
I20260809 02:12:24.214178 19581 ts_manager.cc:194] Registered new tserver with Master: 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001)
I20260809 02:12:24.214569 19581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:33375
I20260809 02:12:24.215024 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:12:24.222451 19621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.216442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33148 (local address 127.5.9.190:45531)
0809 02:12:24.216482 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:24.216484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:24.216593 (+ 109us) server_negotiation.cc:407] Connection header received
0809 02:12:24.216637 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:24.216637 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:24.216646 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:24.216695 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:24.217223 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.217765 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.218495 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.218675 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.218725 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.220002 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:24.220003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:24.220003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:24.220009 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:24.221251 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:24.221596 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:24.221597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:24.221597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:24.221704 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:24.221887 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:24.221888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:24.221888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:24.222094 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:24.222183 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:24.222281 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 02:12:24.222374 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 02:12:24.224177 19581 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:12:24.225801 19581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:12:24.232679 20070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.228272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:41507 (local address 127.0.0.1:54636)
0809 02:12:24.228663 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:24.228674 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:24.228726 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:24.229225 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:24.229225 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:24.229227 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:24.229285 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.229287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.230374 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.230375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.231643 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.231646 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.232345 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.232346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.232430 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.232444 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 02:12:24.232512 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 02:12:24.232576 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":29,"thread_start_us":100,"threads_started":1}
I20260809 02:12:24.232795 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.228268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:33231 (local address 127.0.0.1:49432)
0809 02:12:24.228394 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:24.228438 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:24.228500 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:24.228767 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:24.228771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:24.228774 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:24.228859 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.228861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.230323 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.230324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.231496 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.231498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.232547 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.232548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.232613 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.232617 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:12:24.232670 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 02:12:24.232722 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":62,"thread_start_us":75,"threads_started":1}
I20260809 02:12:24.232849 20071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.228378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49432 (local address 127.5.9.130:33231)
0809 02:12:24.228590 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:24.228595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:24.228606 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:12:24.228651 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:24.228654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:24.228670 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:24.228758 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:24.228964 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.230203 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.231617 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.232357 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.232445 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.232507 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:24.232716 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 02:12:24.232760 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":92,"threads_started":1}
I20260809 02:12:24.233031 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.228730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50600 (local address 127.5.9.131:46001)
0809 02:12:24.228960 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:24.228964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:24.229150 (+ 186us) server_negotiation.cc:407] Connection header received
0809 02:12:24.229202 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:24.229204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:24.229219 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:24.229303 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:24.229403 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.230504 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.231762 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.232449 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.232512 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.232561 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:24.232863 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 02:12:24.232922 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":126,"threads_started":1}
I20260809 02:12:24.233244 20073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.228498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:46001 (local address 127.0.0.1:50600)
0809 02:12:24.229073 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:24.229083 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:24.229138 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:24.229299 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:24.229300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:24.229301 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:24.229336 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.229337 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.230621 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.230622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.231681 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.231683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.232522 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.232523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.232579 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.232744 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 02:12:24.233138 (+ 394us) client_negotiation.cc:240] Negotiation successful
0809 02:12:24.233185 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":85,"threads_started":1}
I20260809 02:12:24.233708 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.228488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54636 (local address 127.5.9.129:41507)
0809 02:12:24.228895 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:24.228899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:24.228908 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:12:24.228948 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:24.228950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:24.228965 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:24.229364 (+ 399us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:24.229453 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.230230 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.231750 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.232245 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.232345 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.232413 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:24.233543 (+ 1130us) server_negotiation.cc:299] Negotiation successful
0809 02:12:24.233596 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":80,"threads_started":1}
I20260809 02:12:24.233618 19851 tablet_service.cc:1467] Processing CreateTablet for tablet 78791ee8cee642b084b1fb91054ccb95 (DEFAULT_TABLE table=test-table [id=312cc0d5014c4afb9a6d35e276a43de8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:12:24.234071 19851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78791ee8cee642b084b1fb91054ccb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:24.234035 19991 tablet_service.cc:1467] Processing CreateTablet for tablet 78791ee8cee642b084b1fb91054ccb95 (DEFAULT_TABLE table=test-table [id=312cc0d5014c4afb9a6d35e276a43de8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:12:24.234356 19991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78791ee8cee642b084b1fb91054ccb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:24.234364 19712 tablet_service.cc:1467] Processing CreateTablet for tablet 78791ee8cee642b084b1fb91054ccb95 (DEFAULT_TABLE table=test-table [id=312cc0d5014c4afb9a6d35e276a43de8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:12:24.234681 19712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78791ee8cee642b084b1fb91054ccb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:24.237308 20075 tablet_bootstrap.cc:492] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: Bootstrap starting.
I20260809 02:12:24.237836 20077 tablet_bootstrap.cc:492] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413: Bootstrap starting.
I20260809 02:12:24.237872 20076 tablet_bootstrap.cc:492] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601: Bootstrap starting.
I20260809 02:12:24.238255 20075 tablet_bootstrap.cc:654] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:24.238520 20076 tablet_bootstrap.cc:654] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:24.238546 20077 tablet_bootstrap.cc:654] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:24.238909 20075 log.cc:824] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:24.239066 20076 log.cc:824] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:24.239087 20077 log.cc:824] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:24.239818 20076 tablet_bootstrap.cc:492] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601: No bootstrap required, opened a new log
I20260809 02:12:24.239813 20075 tablet_bootstrap.cc:492] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: No bootstrap required, opened a new log
I20260809 02:12:24.239879 20077 tablet_bootstrap.cc:492] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413: No bootstrap required, opened a new log
I20260809 02:12:24.239905 20075 ts_tablet_manager.cc:1397] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 02:12:24.239912 20076 ts_tablet_manager.cc:1397] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:12:24.239939 20077 ts_tablet_manager.cc:1397] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:12:24.241562 20076 raft_consensus.cc:348] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } }
I20260809 02:12:24.241801 20076 raft_consensus.cc:374] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:24.241852 20076 raft_consensus.cc:729] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3e4281cf15481197e7667321267601, State: Initialized, Role: FOLLOWER
I20260809 02:12:24.242000 20076 consensus_queue.cc:260] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } }
I20260809 02:12:24.241971 20075 raft_consensus.cc:348] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } }
I20260809 02:12:24.241971 20077 raft_consensus.cc:348] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } }
I20260809 02:12:24.242147 20077 raft_consensus.cc:374] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:24.242147 20075 raft_consensus.cc:374] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:24.242187 20077 raft_consensus.cc:729] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ef8f93814a4ec19caf2e766e221413, State: Initialized, Role: FOLLOWER
I20260809 02:12:24.242187 20075 raft_consensus.cc:729] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d53e1c10ec94ea3b47bcd94c414bb97, State: Initialized, Role: FOLLOWER
I20260809 02:12:24.242321 20076 ts_tablet_manager.cc:1428] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 02:12:24.242306 20075 consensus_queue.cc:260] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } }
I20260809 02:12:24.242307 20077 consensus_queue.cc:260] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } }
I20260809 02:12:24.242487 19919 heartbeater.cc:503] Master 127.5.9.190:45531 was elected leader, sending a full tablet report...
I20260809 02:12:24.242580 20075 ts_tablet_manager.cc:1428] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 02:12:24.242635 20077 ts_tablet_manager.cc:1428] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:12:24.242746 19781 heartbeater.cc:503] Master 127.5.9.190:45531 was elected leader, sending a full tablet report...
I20260809 02:12:24.243026 20057 heartbeater.cc:503] Master 127.5.9.190:45531 was elected leader, sending a full tablet report...
W20260809 02:12:24.294456 19920 tablet.cc:2369] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:12:24.380460 19782 tablet.cc:2369] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:12:24.452867 20058 tablet.cc:2369] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:12:24.459582 20083 raft_consensus.cc:482] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:12:24.459769 20083 raft_consensus.cc:504] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } }
I20260809 02:12:24.460177 20083 leader_election.cc:288] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231), 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001)
I20260809 02:12:24.463332 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.460426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:33231 (local address 127.5.9.129:36127)
0809 02:12:24.460547 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:24.460560 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:24.460629 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:24.460824 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:24.460826 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:24.460851 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:24.460928 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.460930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.461603 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.461604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.462463 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.462466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.463041 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.463042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.463120 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.463123 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:12:24.463184 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 02:12:24.463234 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260809 02:12:24.463321 20071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.460509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:36127 (local address 127.5.9.130:33231)
0809 02:12:24.460580 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:24.460582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:24.460677 (+ 95us) server_negotiation.cc:407] Connection header received
0809 02:12:24.460714 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:24.460715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:24.460728 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:24.460799 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:24.461009 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.461503 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.462565 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.462951 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.463037 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.463048 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:24.463204 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 02:12:24.463253 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:12:24.463498 20085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.460621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:46001 (local address 127.5.9.129:44203)
0809 02:12:24.460737 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:24.460774 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:24.460823 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:24.460978 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:24.460979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:24.460981 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:24.461074 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.461076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.461882 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.461883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.462640 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.462641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.463209 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.463209 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.463289 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.463292 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:12:24.463367 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 02:12:24.463409 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":48,"threads_started":1}
I20260809 02:12:24.463523 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.460728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:44203 (local address 127.5.9.131:46001)
0809 02:12:24.460799 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:24.460801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:24.460812 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:12:24.460881 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:24.460882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:24.460894 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:24.460959 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:24.461157 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.461803 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.462736 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.463126 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.463206 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.463216 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:24.463384 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 02:12:24.463454 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:12:24.463879 20011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78791ee8cee642b084b1fb91054ccb95" candidate_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92ef8f93814a4ec19caf2e766e221413" is_pre_election: true
I20260809 02:12:24.463924 19872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78791ee8cee642b084b1fb91054ccb95" candidate_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3e4281cf15481197e7667321267601" is_pre_election: true
I20260809 02:12:24.464068 19872 raft_consensus.cc:2454] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d53e1c10ec94ea3b47bcd94c414bb97 in term 0.
I20260809 02:12:24.464068 20011 raft_consensus.cc:2454] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d53e1c10ec94ea3b47bcd94c414bb97 in term 0.
I20260809 02:12:24.464321 19663 leader_election.cc:302] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3e4281cf15481197e7667321267601, 7d53e1c10ec94ea3b47bcd94c414bb97; no voters:
I20260809 02:12:24.464462 20083 raft_consensus.cc:2781] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:12:24.464550 20083 raft_consensus.cc:482] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:12:24.464586 20083 raft_consensus.cc:3037] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:24.465385 20083 raft_consensus.cc:504] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } }
I20260809 02:12:24.465541 20083 leader_election.cc:288] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231), 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001)
I20260809 02:12:24.465780 20011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78791ee8cee642b084b1fb91054ccb95" candidate_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92ef8f93814a4ec19caf2e766e221413"
I20260809 02:12:24.465797 19872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78791ee8cee642b084b1fb91054ccb95" candidate_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3e4281cf15481197e7667321267601"
I20260809 02:12:24.465864 20011 raft_consensus.cc:3037] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:24.465898 19872 raft_consensus.cc:3037] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:24.466638 19872 raft_consensus.cc:2454] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d53e1c10ec94ea3b47bcd94c414bb97 in term 1.
I20260809 02:12:24.466634 20011 raft_consensus.cc:2454] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d53e1c10ec94ea3b47bcd94c414bb97 in term 1.
I20260809 02:12:24.466831 19663 leader_election.cc:302] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3e4281cf15481197e7667321267601, 7d53e1c10ec94ea3b47bcd94c414bb97; no voters:
I20260809 02:12:24.466928 20083 raft_consensus.cc:2781] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:12:24.467083 20083 raft_consensus.cc:686] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 LEADER]: Becoming Leader. State: Replica: 7d53e1c10ec94ea3b47bcd94c414bb97, State: Running, Role: LEADER
I20260809 02:12:24.467191 20083 consensus_queue.cc:237] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } }
I20260809 02:12:24.468129 19580 catalog_manager.cc:5495] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d53e1c10ec94ea3b47bcd94c414bb97 (127.5.9.129). New cstate: current_term: 1 leader_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } health_report { overall_health: UNKNOWN } } }
I20260809 02:12:24.517572 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.515068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54648 (local address 127.5.9.129:41507)
0809 02:12:24.515114 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:24.515116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:24.515125 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:12:24.515181 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:24.515182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:24.515204 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:24.515283 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:12:24.515405 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.515917 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.516732 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.516951 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.517063 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.517219 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:12:24.517327 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:12:24.517393 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:24.517432 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 02:12:24.517461 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:12:24.519682 19872 raft_consensus.cc:1261] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601 [term 1 FOLLOWER]: Refusing update from remote peer 7d53e1c10ec94ea3b47bcd94c414bb97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:12:24.519696 20011 raft_consensus.cc:1261] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413 [term 1 FOLLOWER]: Refusing update from remote peer 7d53e1c10ec94ea3b47bcd94c414bb97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:12:24.520087 20083 consensus_queue.cc:1035] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:12:24.520251 20086 consensus_queue.cc:1035] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:12:24.522890 20091 mvcc.cc:205] Tried to move back new op lower bound from 7316445366349815808 to 7316445366138359808. Current Snapshot: MvccSnapshot[applied={T|T < 7316445366349815808}]
I20260809 02:12:24.523188 20090 mvcc.cc:205] Tried to move back new op lower bound from 7316445366349815808 to 7316445366138359808. Current Snapshot: MvccSnapshot[applied={T|T < 7316445366349815808}]
I20260809 02:12:24.525110 19581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:12:24.525223 19581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:12:24.526063 19581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: Sending DeleteTablet for 3 replicas of tablet 78791ee8cee642b084b1fb91054ccb95
I20260809 02:12:24.526436 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 19646
I20260809 02:12:24.526463 19851 tablet_service.cc:1514] Processing DeleteTablet for tablet 78791ee8cee642b084b1fb91054ccb95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:12:24 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:49432
I20260809 02:12:24.526510 19712 tablet_service.cc:1514] Processing DeleteTablet for tablet 78791ee8cee642b084b1fb91054ccb95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:12:24 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:54636
I20260809 02:12:24.527339 19991 tablet_service.cc:1514] Processing DeleteTablet for tablet 78791ee8cee642b084b1fb91054ccb95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:12:24 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:50600
I20260809 02:12:24.527657 20101 tablet_replica.cc:330] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413: stopping tablet replica
I20260809 02:12:24.527752 20101 raft_consensus.cc:2229] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:12:24.527846 20101 raft_consensus.cc:2258] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:12:24.528370 20101 ts_tablet_manager.cc:1905] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:12:24.528631 20100 tablet_replica.cc:330] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601: stopping tablet replica
I20260809 02:12:24.528724 20100 raft_consensus.cc:2229] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:12:24.528813 20100 raft_consensus.cc:2258] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:12:24.529258 20100 ts_tablet_manager.cc:1905] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:12:24.530234 20101 ts_tablet_manager.cc:1918] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:12:24.530308 20101 log.cc:1196] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/78791ee8cee642b084b1fb91054ccb95
I20260809 02:12:24.530592 20101 ts_tablet_manager.cc:1939] T 78791ee8cee642b084b1fb91054ccb95 P 92ef8f93814a4ec19caf2e766e221413: Deleting consensus metadata
I20260809 02:12:24.530936 20100 ts_tablet_manager.cc:1918] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:12:24.531005 20100 log.cc:1196] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/78791ee8cee642b084b1fb91054ccb95
I20260809 02:12:24.531028 19566 catalog_manager.cc:4841] TS 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001): tablet 78791ee8cee642b084b1fb91054ccb95 (table test-table [id=312cc0d5014c4afb9a6d35e276a43de8]) successfully deleted
I20260809 02:12:24.531252 20100 ts_tablet_manager.cc:1939] T 78791ee8cee642b084b1fb91054ccb95 P 3c3e4281cf15481197e7667321267601: Deleting consensus metadata
I20260809 02:12:24.531656 19566 catalog_manager.cc:4841] TS 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231): tablet 78791ee8cee642b084b1fb91054ccb95 (table test-table [id=312cc0d5014c4afb9a6d35e276a43de8]) successfully deleted
W20260809 02:12:24.534706 19565 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:41507 (error 108)
I20260809 02:12:24.535275 20073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.535114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:41507 (local address 127.0.0.1:54654)
0809 02:12:24.535189 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:24.535235 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:41507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 02:12:24.535382 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 19786
W20260809 02:12:24.535441 19565 catalog_manager.cc:4584] TS 7d53e1c10ec94ea3b47bcd94c414bb97 (127.5.9.129:41507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 78791ee8cee642b084b1fb91054ccb95: Network error: Client connection negotiation failed: client connection to 127.5.9.129:41507: connect: Connection refused (error 111)
I20260809 02:12:24.542158 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 19925
I20260809 02:12:24.548652 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 19548
I20260809 02:12:24.555506 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:45531
--webserver_interface=127.5.9.190
--webserver_port=40169
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:45531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:24.642887 20103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:24.643123 20103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:24.643150 20103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:24.644423 20103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:12:24.644508 20103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:24.644531 20103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:24.644553 20103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:12:24.644582 20103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:12:24.646225 20103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:45531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:45531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=40169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:24.646549 20103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:24.646824 20103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:24.658022 20112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:24.658123 20103 server_base.cc:1034] running on GCE node
W20260809 02:12:24.658022 20110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:24.658023 20109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:24.658571 20103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:24.658928 20103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:24.660127 20103 hybrid_clock.cc:648] HybridClock initialized: now 1786241544660166 us; error 95 us; skew 500 ppm
Aug 09 02:12:24 dist-test-slave-p3t5 krb5kdc[19533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241544, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:24.662964 20103 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:12:24.663484 20103 webserver.cc:460] Webserver started at http://127.5.9.190:40169/ using document root <none> and password file <none>
I20260809 02:12:24.663779 20103 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:24.663843 20103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:24.665488 20103 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:12:24.666337 20119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:24.666565 20103 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:12:24.666682 20103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "c34349d7c2bc42d5b6d3c548d6521f05"
format_stamp: "Formatted at 2026-08-09 02:12:23 on dist-test-slave-p3t5"
server_key: "92f314b4514248be5f67bb2fafd23d5b"
server_key_iv: "ea65473e18079b7b046e9b3027c0cfbe"
server_key_version: "encryptionkey@0"
I20260809 02:12:24.667198 20103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:24.685873 20103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:24.688036 20103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:24.688198 20103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:24.692726 20103 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:45531
I20260809 02:12:24.692868 20174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:45531 every 8 connection(s)
I20260809 02:12:24.693220 20103 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:12:24.694839 20175 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:12:24.695447 20175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: Bootstrap starting.
I20260809 02:12:24.702636 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 20103
I20260809 02:12:24.703017 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:41507
--local_ip_for_outbound_sockets=127.5.9.129
--tserver_master_addrs=127.5.9.190:45531
--webserver_port=40259
--webserver_interface=127.5.9.129
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:12:24.704783 20175 log.cc:824] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:24.707420 20175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:12:24.707779 20175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: Bootstrap complete.
I20260809 02:12:24.710407 20175 raft_consensus.cc:348] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } }
I20260809 02:12:24.710664 20175 raft_consensus.cc:729] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c34349d7c2bc42d5b6d3c548d6521f05, State: Initialized, Role: FOLLOWER
I20260809 02:12:24.710842 20175 consensus_queue.cc:260] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } }
I20260809 02:12:24.710940 20175 raft_consensus.cc:388] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:12:24.711024 20175 raft_consensus.cc:482] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:12:24.711103 20175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:12:24.712236 20175 raft_consensus.cc:504] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } }
I20260809 02:12:24.712371 20175 leader_election.cc:302] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c34349d7c2bc42d5b6d3c548d6521f05; no voters:
I20260809 02:12:24.712605 20175 leader_election.cc:288] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 02:12:24.712648 20179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:12:24.712762 20179 raft_consensus.cc:686] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 2 LEADER]: Becoming Leader. State: Replica: c34349d7c2bc42d5b6d3c548d6521f05, State: Running, Role: LEADER
I20260809 02:12:24.712908 20175 sys_catalog.cc:564] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:12:24.712915 20179 consensus_queue.cc:237] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } }
I20260809 02:12:24.713250 20179 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c34349d7c2bc42d5b6d3c548d6521f05. Latest consensus state: current_term: 2 leader_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } } }
I20260809 02:12:24.713323 20179 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [sys.catalog]: This master's current role is: LEADER
I20260809 02:12:24.713497 20180 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } } }
I20260809 02:12:24.713577 20180 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [sys.catalog]: This master's current role is: LEADER
I20260809 02:12:24.713725 20184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:12:24.714715 20184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:12:24.714844 20184 catalog_manager.cc:1259] Loaded cluster ID: 3f04f1a79884492ca17b13b0718aeea3
I20260809 02:12:24.714891 20184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:12:24.715201 20184 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 02:12:24.718257 20184 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 02:12:24.722421 20184 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:12:24.722654 20184 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: Loaded TSK: 0
I20260809 02:12:24.722899 20184 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:12:24.803458 20177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:24.803746 20177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:24.803791 20177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:24.805058 20177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:24.805131 20177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:24.805181 20177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:12:24.806910 20177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:41507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=40259
--tserver_master_addrs=127.5.9.190:45531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:24.807231 20177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:24.807492 20177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:24.812984 20202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:24.813031 20205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:24.813031 20203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:24.814203 20177 server_base.cc:1034] running on GCE node
I20260809 02:12:24.814476 20177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:24.814792 20177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:24.815991 20177 hybrid_clock.cc:648] HybridClock initialized: now 1786241544815971 us; error 36 us; skew 500 ppm
Aug 09 02:12:24 dist-test-slave-p3t5 krb5kdc[19533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241544, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:24.819070 20177 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:12:24.819710 20177 webserver.cc:460] Webserver started at http://127.5.9.129:40259/ using document root <none> and password file <none>
I20260809 02:12:24.819968 20177 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:24.820056 20177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:24.822082 20177 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:12:24.822875 20212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:24.823136 20177 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:12:24.823258 20177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "7d53e1c10ec94ea3b47bcd94c414bb97"
format_stamp: "Formatted at 2026-08-09 02:12:23 on dist-test-slave-p3t5"
server_key: "49a225c8f0824f3017f1f87fffd8646e"
server_key_iv: "bcf460ef7907bf90aa83774f231494d9"
server_key_version: "encryptionkey@0"
I20260809 02:12:24.823757 20177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:24.844689 20177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:24.847571 20177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:24.847744 20177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:24.848030 20177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:24.848595 20219 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:12:24.850337 20177 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:12:24.850409 20177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 02:12:24.850448 20177 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:12:24.851368 20177 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:12:24.851418 20177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 02:12:24.851519 20219 tablet_bootstrap.cc:492] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: Bootstrap starting.
I20260809 02:12:24.858752 20177 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:41507
I20260809 02:12:24.859092 20329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:41507 every 8 connection(s)
I20260809 02:12:24.859205 20177 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:12:24.860352 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 20177
I20260809 02:12:24.860728 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:33231
--local_ip_for_outbound_sockets=127.5.9.130
--tserver_master_addrs=127.5.9.190:45531
--webserver_port=43817
--webserver_interface=127.5.9.130
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:12:24.863672 20219 log.cc:824] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:24.868104 20219 tablet_bootstrap.cc:492] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:12:24.868475 20219 tablet_bootstrap.cc:492] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: Bootstrap complete.
I20260809 02:12:24.869918 20219 ts_tablet_manager.cc:1397] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.014s
Aug 09 02:12:24 dist-test-slave-p3t5 krb5kdc[19533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241544, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:24.872149 20219 raft_consensus.cc:348] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } }
I20260809 02:12:24.872594 20219 raft_consensus.cc:729] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d53e1c10ec94ea3b47bcd94c414bb97, State: Initialized, Role: FOLLOWER
I20260809 02:12:24.872790 20219 consensus_queue.cc:260] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } }
I20260809 02:12:24.873136 20219 ts_tablet_manager.cc:1428] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 02:12:24.877332 20333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.860101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:48869 (local address 127.5.9.190:45531)
0809 02:12:24.861904 (+ 1803us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:24.861915 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:24.863796 (+ 1881us) server_negotiation.cc:407] Connection header received
0809 02:12:24.864241 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:24.864248 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:24.864311 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:24.864459 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:24.865308 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.866869 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.868673 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.869367 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.869503 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.872858 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:24.872862 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:24.872862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:24.872936 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:24.875080 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:24.875709 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:24.875712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:24.875713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:24.875834 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:24.876163 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:24.876164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:24.876164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:24.876513 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:24.876605 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:24.876996 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 02:12:24.877143 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1743,"thread_start_us":91,"threads_started":1}
I20260809 02:12:24.877336 20332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.859992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:48869)
0809 02:12:24.862955 (+ 2963us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:24.863041 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:24.863939 (+ 898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:24.864522 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:24.864527 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:24.864896 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:24.865155 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.865162 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.867006 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.867007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.868476 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.868481 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.869665 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.869666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.869820 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.870619 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:24.870631 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:24.872731 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:24.875190 (+ 2459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:24.875199 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:24.875202 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:24.875599 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:24.875914 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:24.875915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:24.875915 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:24.876082 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:24.876601 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:24.876610 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:24.876802 (+ 192us) client_negotiation.cc:769] Sending connection context
0809 02:12:24.876928 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 02:12:24.877120 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2902,"thread_start_us":121,"threads_started":1}
I20260809 02:12:24.879060 20330 heartbeater.cc:347] Connected to a master server at 127.5.9.190:45531
I20260809 02:12:24.879173 20330 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:24.879385 20330 heartbeater.cc:511] Master 127.5.9.190:45531 requested a full tablet report, sending...
I20260809 02:12:24.880157 20137 ts_manager.cc:194] Registered new tserver with Master: 7d53e1c10ec94ea3b47bcd94c414bb97 (127.5.9.129:41507)
I20260809 02:12:24.881268 20137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:48869
I20260809 02:12:24.882287 20330 heartbeater.cc:503] Master 127.5.9.190:45531 was elected leader, sending a full tablet report...
Aug 09 02:12:24 dist-test-slave-p3t5 krb5kdc[19533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241544, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for kudu/127.5.9.129@KRBTEST.COM
I20260809 02:12:24.889963 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.880892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54668 (local address 127.5.9.129:41507)
0809 02:12:24.881095 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:24.881100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:24.881219 (+ 119us) server_negotiation.cc:407] Connection header received
0809 02:12:24.881320 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:24.881322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:24.881343 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:24.881414 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:24.881971 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.882799 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.883918 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.884213 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.884277 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.886732 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:24.886742 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:24.886742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:24.886750 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:24.888402 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:24.888896 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:24.888900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:24.888900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:24.888975 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:24.889236 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:24.889237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:24.889237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:24.889421 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:24.889510 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:24.889779 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 02:12:24.889880 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":112,"threads_started":1}
I20260809 02:12:24.889993 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:24.880789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:41507 (local address 127.0.0.1:54668)
0809 02:12:24.881108 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:24.881146 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:24.881230 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:24.881470 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:24.881475 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:24.881833 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:24.881883 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.881885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.882867 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.882867 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.883824 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:24.883828 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:24.884300 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:24.884301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:24.884392 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:24.884919 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:24.884925 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:24.886623 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:24.888492 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:24.888500 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:24.888503 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:24.888790 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:24.889057 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:24.889057 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:24.889058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:24.889136 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:24.889507 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:24.889514 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:24.889579 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 02:12:24.889743 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 02:12:24.889891 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":113,"threads_started":1}
I20260809 02:12:24.890458 20263 tablet_service.cc:1514] Processing DeleteTablet for tablet 78791ee8cee642b084b1fb91054ccb95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:12:24 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:54668
I20260809 02:12:24.891134 20339 tablet_replica.cc:330] stopping tablet replica
I20260809 02:12:24.891265 20339 raft_consensus.cc:2229] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:12:24.891360 20339 raft_consensus.cc:2258] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:12:24.891671 20339 ts_tablet_manager.cc:1905] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:12:24.893994 20339 ts_tablet_manager.cc:1918] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:12:24.894102 20339 log.cc:1196] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal/wals/78791ee8cee642b084b1fb91054ccb95
I20260809 02:12:24.894354 20339 ts_tablet_manager.cc:1939] T 78791ee8cee642b084b1fb91054ccb95 P 7d53e1c10ec94ea3b47bcd94c414bb97: Deleting consensus metadata
I20260809 02:12:24.894934 20120 catalog_manager.cc:4841] TS 7d53e1c10ec94ea3b47bcd94c414bb97 (127.5.9.129:41507): tablet 78791ee8cee642b084b1fb91054ccb95 (table test-table [id=312cc0d5014c4afb9a6d35e276a43de8]) successfully deleted
W20260809 02:12:24.957652 20334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:24.957896 20334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:24.957931 20334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:24.959116 20334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:24.959162 20334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:24.959189 20334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:12:24.960659 20334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:33231
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=43817
--tserver_master_addrs=127.5.9.190:45531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:24.960878 20334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:24.961089 20334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:12:24.967043 20334 server_base.cc:1034] running on GCE node
W20260809 02:12:24.966992 20347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:24.967005 20345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:24.967005 20344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:24.967456 20334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:24.967733 20334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:24.968900 20334 hybrid_clock.cc:648] HybridClock initialized: now 1786241544968887 us; error 33 us; skew 500 ppm
Aug 09 02:12:24 dist-test-slave-p3t5 krb5kdc[19533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241544, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:24.972083 20334 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:12:24.972527 20334 webserver.cc:460] Webserver started at http://127.5.9.130:43817/ using document root <none> and password file <none>
I20260809 02:12:24.972767 20334 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:24.972827 20334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:24.974367 20334 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:12:24.975073 20354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:24.975260 20334 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:12:24.975368 20334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "3c3e4281cf15481197e7667321267601"
format_stamp: "Formatted at 2026-08-09 02:12:24 on dist-test-slave-p3t5"
server_key: "79b0bc1a2234816f701131b3cc3f5cd4"
server_key_iv: "e7b1ff2d86f8cbd7a08923f9a03c7375"
server_key_version: "encryptionkey@0"
I20260809 02:12:24.975762 20334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:24.994987 20334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:24.997424 20334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:24.997663 20334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:24.998008 20334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:24.998358 20334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:12:24.998414 20334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:24.998451 20334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:12:24.998478 20334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:25.004679 20334 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:33231
I20260809 02:12:25.004724 20474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:33231 every 8 connection(s)
I20260809 02:12:25.005226 20334 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:12:25.008726 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 20334
I20260809 02:12:25.009189 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:46001
--local_ip_for_outbound_sockets=127.5.9.131
--tserver_master_addrs=127.5.9.190:45531
--webserver_port=42065
--webserver_interface=127.5.9.131
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:12:25 dist-test-slave-p3t5 krb5kdc[19533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241544, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:25.017779 20333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.006030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:44281 (local address 127.5.9.190:45531)
0809 02:12:25.006107 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:25.006109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:25.006359 (+ 250us) server_negotiation.cc:407] Connection header received
0809 02:12:25.006924 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:25.006926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:25.006944 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:25.007014 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:25.008055 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.008591 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.010122 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.010361 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.010453 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:25.013356 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:25.013359 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:25.013359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:25.013370 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:25.015902 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:25.016556 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:25.016558 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:25.016559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:25.016637 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:25.016938 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:25.016939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:25.016939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:25.017124 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:25.017201 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:25.017481 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 02:12:25.017598 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:12:25.018000 20477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.006029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:44281)
0809 02:12:25.006224 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:25.006264 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:25.006810 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:25.007094 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:25.007098 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:25.007717 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:25.007926 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:25.007931 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.008706 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.008707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:25.010000 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:25.010004 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.010465 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.010467 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:25.010601 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:25.011235 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:25.011246 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:25.013190 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:25.016101 (+ 2911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:25.016113 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:25.016115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:25.016463 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:25.016722 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:25.016722 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:25.016723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:25.016870 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:25.017202 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:25.017207 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:25.017349 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 02:12:25.017433 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:12:25.017659 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":106,"threads_started":1}
I20260809 02:12:25.018527 20475 heartbeater.cc:347] Connected to a master server at 127.5.9.190:45531
I20260809 02:12:25.018657 20475 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:25.018980 20475 heartbeater.cc:511] Master 127.5.9.190:45531 requested a full tablet report, sending...
I20260809 02:12:25.019500 20137 ts_manager.cc:194] Registered new tserver with Master: 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231)
I20260809 02:12:25.019904 20137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:44281
W20260809 02:12:25.097121 20478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:25.097360 20478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:25.097388 20478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:25.098676 20478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:25.098732 20478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:25.098762 20478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:12:25.100325 20478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:46001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=42065
--tserver_master_addrs=127.5.9.190:45531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:25.100605 20478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:25.100862 20478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:25.106089 20483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:25.106089 20487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:25.106099 20485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:25.106371 20478 server_base.cc:1034] running on GCE node
I20260809 02:12:25.106630 20478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:25.106868 20478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:25.108026 20478 hybrid_clock.cc:648] HybridClock initialized: now 1786241545108006 us; error 34 us; skew 500 ppm
Aug 09 02:12:25 dist-test-slave-p3t5 krb5kdc[19533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241545, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:25.110805 20478 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:12:25.111305 20478 webserver.cc:460] Webserver started at http://127.5.9.131:42065/ using document root <none> and password file <none>
I20260809 02:12:25.111569 20478 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:25.111660 20478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:25.113318 20478 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:12:25.114111 20494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:25.114310 20478 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:12:25.114413 20478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "92ef8f93814a4ec19caf2e766e221413"
format_stamp: "Formatted at 2026-08-09 02:12:24 on dist-test-slave-p3t5"
server_key: "620790ddd217977932153341cafb26bf"
server_key_iv: "0f7bc02ae5d03fe1da31b6e4e10a7c83"
server_key_version: "encryptionkey@0"
I20260809 02:12:25.114802 20478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:25.140057 20478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:25.142122 20478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:25.142292 20478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:25.142551 20478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:25.142880 20478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:12:25.142944 20478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:25.142987 20478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:12:25.143028 20478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:25.149194 20478 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:46001
I20260809 02:12:25.149235 20612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:46001 every 8 connection(s)
I20260809 02:12:25.149649 20478 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
Aug 09 02:12:25 dist-test-slave-p3t5 krb5kdc[19533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241545, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:25.156642 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 20478
I20260809 02:12:25.159621 20333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.150456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:40955 (local address 127.5.9.190:45531)
0809 02:12:25.150551 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:25.150553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:25.150738 (+ 185us) server_negotiation.cc:407] Connection header received
0809 02:12:25.151271 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:25.151272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:25.151282 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:25.151340 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:25.152309 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.152836 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.153884 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.154072 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.154126 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:25.156423 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:25.156425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:25.156425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:25.156434 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:25.158099 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:25.158567 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:25.158569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:25.158569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:25.158635 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:25.158925 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:25.158926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:25.158926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:25.159049 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:25.159124 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:25.159426 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 02:12:25.159523 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 02:12:25.159830 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.150422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:40955)
0809 02:12:25.150611 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:25.150652 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:25.151181 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:25.151399 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:25.151403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:25.152015 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:25.152222 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:25.152227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.152954 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.152955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:25.153804 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:25.153806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.154148 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.154148 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:25.154254 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:25.154809 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:25.154818 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:25.156308 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:25.158206 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:25.158215 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:25.158218 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:25.158485 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:25.158721 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:25.158722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:25.158723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:25.158849 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:25.159114 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:25.159122 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:25.159277 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 02:12:25.159390 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 02:12:25.159571 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":110,"threads_started":1}
I20260809 02:12:25.160347 20613 heartbeater.cc:347] Connected to a master server at 127.5.9.190:45531
I20260809 02:12:25.160477 20613 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:25.160678 20613 heartbeater.cc:511] Master 127.5.9.190:45531 requested a full tablet report, sending...
I20260809 02:12:25.161147 20137 ts_manager.cc:194] Registered new tserver with Master: 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001)
I20260809 02:12:25.161700 20137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:40955
I20260809 02:12:25.164083 20617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.157053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33150 (local address 127.5.9.190:45531)
0809 02:12:25.157203 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:25.157205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:25.157231 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:12:25.157318 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:25.157319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:25.157332 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:25.157408 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:25.158053 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.158571 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.159316 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.159523 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.159597 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:25.161023 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:25.161025 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:25.161025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:25.161033 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:25.163021 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:25.163410 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:25.163412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:25.163412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:25.163471 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:25.163623 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:25.163624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:25.163624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:25.163719 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:25.163838 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:25.163923 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 02:12:25.163992 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260809 02:12:25.164763 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:12:25.167747 20617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.165134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33158 (local address 127.5.9.190:45531)
0809 02:12:25.165194 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:25.165197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:25.165297 (+ 100us) server_negotiation.cc:407] Connection header received
0809 02:12:25.165374 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:25.165375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:25.165416 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:25.165500 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:12:25.165649 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.166140 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.166967 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.167183 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.167257 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:25.167420 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:12:25.167531 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:12:25.167624 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:25.167643 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 02:12:25.167667 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 02:12:25.168062 20137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:12:25.169754 20137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:12:25.171922 20263 tablet_service.cc:1467] Processing CreateTablet for tablet 848724802b54487684e67816ea91fa42 (DEFAULT_TABLE table=test-table [id=b241338b7a044279ab2b57ce1e095da1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:12:25.172138 20263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 848724802b54487684e67816ea91fa42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:25.173967 20219 tablet_bootstrap.cc:492] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97: Bootstrap starting.
I20260809 02:12:25.174309 20219 tablet_bootstrap.cc:654] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:25.175544 20219 tablet_bootstrap.cc:492] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97: No bootstrap required, opened a new log
I20260809 02:12:25.175598 20219 ts_tablet_manager.cc:1397] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:12:25.175832 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.171656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:33231 (local address 127.0.0.1:49448)
0809 02:12:25.171693 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:25.171704 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:25.171741 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:25.172459 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:25.172460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:25.172461 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:25.172540 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:25.172541 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.173606 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.173607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:25.174810 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:25.174813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.175551 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.175553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:25.175626 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:25.175654 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 02:12:25.175710 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 02:12:25.175760 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 02:12:25.175796 20219 raft_consensus.cc:348] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } }
I20260809 02:12:25.175864 20621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.171921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49448 (local address 127.5.9.130:33231)
0809 02:12:25.172256 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:25.172262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:25.172274 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:12:25.172332 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:25.172334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:25.172350 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:25.172448 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:25.172646 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.173505 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.174907 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.175462 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.175544 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:25.175616 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:25.175741 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 02:12:25.175790 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":96,"threads_started":1}
I20260809 02:12:25.175874 20219 raft_consensus.cc:374] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:25.175926 20219 raft_consensus.cc:729] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d53e1c10ec94ea3b47bcd94c414bb97, State: Initialized, Role: FOLLOWER
I20260809 02:12:25.175982 20219 consensus_queue.cc:260] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } }
I20260809 02:12:25.176101 20219 ts_tablet_manager.cc:1428] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:12:25.176800 20619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.171758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:46001 (local address 127.0.0.1:50610)
0809 02:12:25.172105 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:25.172115 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:25.172168 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:25.172898 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:25.172898 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:25.172900 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:25.172943 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:25.172944 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.174263 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.174264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:25.175475 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:25.175478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.176529 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.176530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:25.176599 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:25.176603 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:12:25.176684 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 02:12:25.176725 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":68,"threads_started":1}
I20260809 02:12:25.176851 20620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.171865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50610 (local address 127.5.9.131:46001)
0809 02:12:25.172675 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:25.172680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:25.172690 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:12:25.172727 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:25.172729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:25.172745 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:25.172897 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:25.173007 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.174158 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.175597 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.176354 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.176404 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:25.176454 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:25.176705 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 02:12:25.176746 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":135,"threads_started":1}
I20260809 02:12:25.177000 20406 tablet_service.cc:1467] Processing CreateTablet for tablet 848724802b54487684e67816ea91fa42 (DEFAULT_TABLE table=test-table [id=b241338b7a044279ab2b57ce1e095da1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:12:25.177409 20406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 848724802b54487684e67816ea91fa42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:25.178403 20542 tablet_service.cc:1467] Processing CreateTablet for tablet 848724802b54487684e67816ea91fa42 (DEFAULT_TABLE table=test-table [id=b241338b7a044279ab2b57ce1e095da1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:12:25.178820 20542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 848724802b54487684e67816ea91fa42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:25.181254 20623 tablet_bootstrap.cc:492] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601: Bootstrap starting.
I20260809 02:12:25.181964 20623 tablet_bootstrap.cc:654] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:25.182457 20623 log.cc:824] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:25.182487 20624 tablet_bootstrap.cc:492] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413: Bootstrap starting.
I20260809 02:12:25.183190 20624 tablet_bootstrap.cc:654] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:25.183780 20624 log.cc:824] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:25.183890 20623 tablet_bootstrap.cc:492] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601: No bootstrap required, opened a new log
I20260809 02:12:25.183970 20623 ts_tablet_manager.cc:1397] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:12:25.184660 20624 tablet_bootstrap.cc:492] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413: No bootstrap required, opened a new log
I20260809 02:12:25.184716 20624 ts_tablet_manager.cc:1397] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:12:25.185446 20623 raft_consensus.cc:348] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } }
I20260809 02:12:25.185621 20623 raft_consensus.cc:374] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:25.185694 20623 raft_consensus.cc:729] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3e4281cf15481197e7667321267601, State: Initialized, Role: FOLLOWER
I20260809 02:12:25.185832 20623 consensus_queue.cc:260] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } }
I20260809 02:12:25.186162 20623 ts_tablet_manager.cc:1428] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 02:12:25.186192 20475 heartbeater.cc:503] Master 127.5.9.190:45531 was elected leader, sending a full tablet report...
I20260809 02:12:25.186313 20624 raft_consensus.cc:348] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } }
I20260809 02:12:25.186481 20624 raft_consensus.cc:374] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:25.186517 20624 raft_consensus.cc:729] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ef8f93814a4ec19caf2e766e221413, State: Initialized, Role: FOLLOWER
I20260809 02:12:25.186642 20624 consensus_queue.cc:260] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } }
I20260809 02:12:25.186990 20613 heartbeater.cc:503] Master 127.5.9.190:45531 was elected leader, sending a full tablet report...
I20260809 02:12:25.187214 20624 ts_tablet_manager.cc:1428] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 02:12:25.191589 20336 raft_consensus.cc:482] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:12:25.191725 20336 raft_consensus.cc:504] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } }
I20260809 02:12:25.192005 20336 leader_election.cc:288] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231), 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001)
I20260809 02:12:25.195976 20620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.192474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:49623 (local address 127.5.9.131:46001)
0809 02:12:25.192534 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:25.192536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:25.192678 (+ 142us) server_negotiation.cc:407] Connection header received
0809 02:12:25.192729 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:25.192730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:25.192741 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:25.192804 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:25.192981 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.193684 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.194978 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.195555 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.195632 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:25.195643 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:25.195844 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 02:12:25.195890 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 02:12:25.195974 20621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.192322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:47103 (local address 127.5.9.130:33231)
0809 02:12:25.192368 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:25.192369 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:25.192379 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:12:25.192449 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:25.192451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:25.192463 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:25.192519 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:25.192688 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.193472 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.194979 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.195555 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.195632 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:25.195643 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:25.195865 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 02:12:25.195900 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:12:25.195988 20332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.192192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:33231 (local address 127.5.9.129:47103)
0809 02:12:25.192268 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:25.192282 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:25.192358 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:25.192524 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:25.192525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:25.192526 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:25.192605 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:25.192607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.193546 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.193547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:25.194873 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:25.194876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.195642 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.195643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:25.195727 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:25.195758 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 02:12:25.195837 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:12:25.195871 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":1}
I20260809 02:12:25.195997 20629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.192291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:46001 (local address 127.5.9.129:49623)
0809 02:12:25.192606 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:25.192618 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:25.192672 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:25.192812 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:25.192812 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:25.192813 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:25.192870 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:25.192872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.193753 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.193753 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:25.194873 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:25.194876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.195662 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.195663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:25.195727 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:25.195758 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 02:12:25.195825 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:12:25.195868 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":114,"threads_started":1}
I20260809 02:12:25.196446 20564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "848724802b54487684e67816ea91fa42" candidate_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92ef8f93814a4ec19caf2e766e221413" is_pre_election: true
I20260809 02:12:25.196482 20427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "848724802b54487684e67816ea91fa42" candidate_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3e4281cf15481197e7667321267601" is_pre_election: true
I20260809 02:12:25.196612 20564 raft_consensus.cc:2454] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d53e1c10ec94ea3b47bcd94c414bb97 in term 0.
I20260809 02:12:25.196658 20427 raft_consensus.cc:2454] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d53e1c10ec94ea3b47bcd94c414bb97 in term 0.
I20260809 02:12:25.196841 20214 leader_election.cc:302] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d53e1c10ec94ea3b47bcd94c414bb97, 92ef8f93814a4ec19caf2e766e221413; no voters:
I20260809 02:12:25.196982 20336 raft_consensus.cc:2781] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:12:25.197043 20336 raft_consensus.cc:482] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:12:25.197072 20336 raft_consensus.cc:3037] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:25.197851 20336 raft_consensus.cc:504] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } }
I20260809 02:12:25.197999 20336 leader_election.cc:288] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231), 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001)
I20260809 02:12:25.198153 20427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "848724802b54487684e67816ea91fa42" candidate_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3e4281cf15481197e7667321267601"
I20260809 02:12:25.198172 20564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "848724802b54487684e67816ea91fa42" candidate_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92ef8f93814a4ec19caf2e766e221413"
I20260809 02:12:25.198244 20564 raft_consensus.cc:3037] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:25.198253 20427 raft_consensus.cc:3037] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:25.199014 20564 raft_consensus.cc:2454] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d53e1c10ec94ea3b47bcd94c414bb97 in term 1.
I20260809 02:12:25.199016 20427 raft_consensus.cc:2454] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d53e1c10ec94ea3b47bcd94c414bb97 in term 1.
I20260809 02:12:25.199213 20214 leader_election.cc:302] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3e4281cf15481197e7667321267601, 7d53e1c10ec94ea3b47bcd94c414bb97; no voters:
I20260809 02:12:25.199326 20336 raft_consensus.cc:2781] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:12:25.199563 20336 raft_consensus.cc:686] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 LEADER]: Becoming Leader. State: Replica: 7d53e1c10ec94ea3b47bcd94c414bb97, State: Running, Role: LEADER
I20260809 02:12:25.199648 20336 consensus_queue.cc:237] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } }
I20260809 02:12:25.200493 20137 catalog_manager.cc:5495] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d53e1c10ec94ea3b47bcd94c414bb97 (127.5.9.129). New cstate: current_term: 1 leader_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } health_report { overall_health: HEALTHY } } }
I20260809 02:12:25.212244 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.209987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54676 (local address 127.5.9.129:41507)
0809 02:12:25.210048 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:25.210050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:25.210058 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:12:25.210132 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:25.210133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:25.210152 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:25.210211 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:12:25.210329 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.210822 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.211518 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:25.211717 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:25.211766 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:25.211887 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:12:25.212014 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:12:25.212072 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:25.212108 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 02:12:25.212129 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:12:25.214004 20427 raft_consensus.cc:1261] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601 [term 1 FOLLOWER]: Refusing update from remote peer 7d53e1c10ec94ea3b47bcd94c414bb97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:12:25.214016 20564 raft_consensus.cc:1261] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413 [term 1 FOLLOWER]: Refusing update from remote peer 7d53e1c10ec94ea3b47bcd94c414bb97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:12:25.214398 20336 consensus_queue.cc:1035] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:12:25.214588 20630 consensus_queue.cc:1035] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:12:25.217046 20632 mvcc.cc:205] Tried to move back new op lower bound from 7316445369193672704 to 7316445369138487296. Current Snapshot: MvccSnapshot[applied={T|T < 7316445369193672704}]
I20260809 02:12:25.217846 20634 mvcc.cc:205] Tried to move back new op lower bound from 7316445369193672704 to 7316445369138487296. Current Snapshot: MvccSnapshot[applied={T|T < 7316445369193672704 or (T in {7316445369193672704})}]
I20260809 02:12:25.219374 20136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:12:25.219482 20136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:12:25.220324 20136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: Sending DeleteTablet for 3 replicas of tablet 848724802b54487684e67816ea91fa42
I20260809 02:12:25.220681 20263 tablet_service.cc:1514] Processing DeleteTablet for tablet 848724802b54487684e67816ea91fa42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:12:25 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:54668
I20260809 02:12:25.220722 20406 tablet_service.cc:1514] Processing DeleteTablet for tablet 848724802b54487684e67816ea91fa42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:12:25 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:49448
I20260809 02:12:25.220803 20339 tablet_replica.cc:330] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97: stopping tablet replica
I20260809 02:12:25.220876 20339 raft_consensus.cc:2229] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:12:25.220958 20339 raft_consensus.cc:2258] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:12:25.220921 20542 tablet_service.cc:1514] Processing DeleteTablet for tablet 848724802b54487684e67816ea91fa42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:12:25 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:50610
I20260809 02:12:25.220988 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 20177
I20260809 02:12:25.225831 20642 tablet_replica.cc:330] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601: stopping tablet replica
I20260809 02:12:25.226012 20642 raft_consensus.cc:2229] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:12:25.226094 20643 tablet_replica.cc:330] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413: stopping tablet replica
I20260809 02:12:25.226162 20642 raft_consensus.cc:2258] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:12:25.226212 20643 raft_consensus.cc:2229] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:12:25.226330 20643 raft_consensus.cc:2258] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:12:25.226716 20642 ts_tablet_manager.cc:1905] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:12:25.226868 20643 ts_tablet_manager.cc:1905] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:12:25.228392 20642 ts_tablet_manager.cc:1918] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:12:25.228492 20642 log.cc:1196] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/848724802b54487684e67816ea91fa42
I20260809 02:12:25.228765 20642 ts_tablet_manager.cc:1939] T 848724802b54487684e67816ea91fa42 P 3c3e4281cf15481197e7667321267601: Deleting consensus metadata
I20260809 02:12:25.228912 20643 ts_tablet_manager.cc:1918] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:12:25.228991 20643 log.cc:1196] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/848724802b54487684e67816ea91fa42
W20260809 02:12:25.229189 20120 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:41507 (error 108)
I20260809 02:12:25.229285 20643 ts_tablet_manager.cc:1939] T 848724802b54487684e67816ea91fa42 P 92ef8f93814a4ec19caf2e766e221413: Deleting consensus metadata
I20260809 02:12:25.229550 20121 catalog_manager.cc:4841] TS 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231): tablet 848724802b54487684e67816ea91fa42 (table test-table [id=b241338b7a044279ab2b57ce1e095da1]) successfully deleted
I20260809 02:12:25.229841 20121 catalog_manager.cc:4841] TS 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001): tablet 848724802b54487684e67816ea91fa42 (table test-table [id=b241338b7a044279ab2b57ce1e095da1]) successfully deleted
W20260809 02:12:25.229867 20619 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:12:25.229689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:41507 (local address 127.0.0.1:54682)
0809 02:12:25.229761 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:25.229817 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:41507: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 02:12:25.230166 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 20334
W20260809 02:12:25.229998 20120 catalog_manager.cc:4584] TS 7d53e1c10ec94ea3b47bcd94c414bb97 (127.5.9.129:41507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 848724802b54487684e67816ea91fa42: Network error: Client connection negotiation failed: client connection to 127.5.9.129:41507: connect: Connection reset by peer (error 104)
I20260809 02:12:25.236544 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 20478
I20260809 02:12:25.242328 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 20103
I20260809 02:12:25.249855 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:45531
--webserver_interface=127.5.9.190
--webserver_port=40169
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:45531
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:25.337440 20645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:25.337769 20645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:25.337814 20645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:25.339109 20645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:12:25.339191 20645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:25.339218 20645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:25.339246 20645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:12:25.339274 20645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:12:25.340906 20645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:45531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:45531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=40169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:25.341233 20645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:25.341507 20645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:25.346642 20650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:25.346690 20653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:25.346815 20645 server_base.cc:1034] running on GCE node
W20260809 02:12:25.346688 20651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:25.347235 20645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:25.347554 20645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:25.348707 20645 hybrid_clock.cc:648] HybridClock initialized: now 1786241545348674 us; error 45 us; skew 500 ppm
Aug 09 02:12:25 dist-test-slave-p3t5 krb5kdc[19533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241545, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:25.351814 20645 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:12:25.352388 20645 webserver.cc:460] Webserver started at http://127.5.9.190:40169/ using document root <none> and password file <none>
I20260809 02:12:25.352665 20645 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:25.352715 20645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:25.354244 20645 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:12:25.355124 20660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:25.355348 20645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:25.355443 20645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "c34349d7c2bc42d5b6d3c548d6521f05"
format_stamp: "Formatted at 2026-08-09 02:12:23 on dist-test-slave-p3t5"
server_key: "92f314b4514248be5f67bb2fafd23d5b"
server_key_iv: "ea65473e18079b7b046e9b3027c0cfbe"
server_key_version: "encryptionkey@0"
I20260809 02:12:25.355859 20645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:25.368681 20645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:25.370925 20645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:25.371089 20645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:25.375622 20645 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:45531
I20260809 02:12:25.375658 20714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:45531 every 8 connection(s)
I20260809 02:12:25.376091 20645 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:12:25.377920 20715 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:12:25.378571 20715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: Bootstrap starting.
I20260809 02:12:25.385861 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 20645
I20260809 02:12:25.386178 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:41507
--local_ip_for_outbound_sockets=127.5.9.129
--tserver_master_addrs=127.5.9.190:45531
--webserver_port=40259
--webserver_interface=127.5.9.129
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:12:25.387698 20715 log.cc:824] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:25.390652 20715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:12:25.391014 20715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: Bootstrap complete.
I20260809 02:12:25.393532 20715 raft_consensus.cc:348] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } }
I20260809 02:12:25.393918 20715 raft_consensus.cc:729] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c34349d7c2bc42d5b6d3c548d6521f05, State: Initialized, Role: FOLLOWER
I20260809 02:12:25.394140 20715 consensus_queue.cc:260] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } }
I20260809 02:12:25.394241 20715 raft_consensus.cc:388] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:12:25.394299 20715 raft_consensus.cc:482] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:12:25.394354 20715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 2 FOLLOWER]: Advancing to term 3
I20260809 02:12:25.395498 20715 raft_consensus.cc:504] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } }
I20260809 02:12:25.395637 20715 leader_election.cc:302] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c34349d7c2bc42d5b6d3c548d6521f05; no voters:
I20260809 02:12:25.395872 20715 leader_election.cc:288] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 02:12:25.395920 20719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 02:12:25.396050 20719 raft_consensus.cc:686] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [term 3 LEADER]: Becoming Leader. State: Replica: c34349d7c2bc42d5b6d3c548d6521f05, State: Running, Role: LEADER
I20260809 02:12:25.396152 20719 consensus_queue.cc:237] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } }
I20260809 02:12:25.396184 20715 sys_catalog.cc:564] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:12:25.396477 20719 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c34349d7c2bc42d5b6d3c548d6521f05. Latest consensus state: current_term: 3 leader_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } } }
I20260809 02:12:25.396569 20719 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [sys.catalog]: This master's current role is: LEADER
I20260809 02:12:25.396486 20720 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34349d7c2bc42d5b6d3c548d6521f05" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45531 } } }
I20260809 02:12:25.396756 20720 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05 [sys.catalog]: This master's current role is: LEADER
I20260809 02:12:25.398298 20735 catalog_manager.cc:1259] Loaded cluster ID: 3f04f1a79884492ca17b13b0718aeea3
I20260809 02:12:25.398362 20735 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: loading cluster ID for follower catalog manager: success
W20260809 02:12:25.398663 20735 catalog_manager.cc:1576] T 00000000000000000000000000000000 P c34349d7c2bc42d5b6d3c548d6521f05: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260809 02:12:25.398734 20735 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260809 02:12:25.398774 20723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:12:25.399161 20723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:12:25.399274 20723 catalog_manager.cc:1259] Loaded cluster ID: 3f04f1a79884492ca17b13b0718aeea3
I20260809 02:12:25.399329 20723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 02:12:25.399477 20723 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f230f6239ad google::LogMessage::Fail() at ??:0
@ 0x7f230f62688a google::LogMessage::SendToLog() at ??:0
@ 0x7f230f6234de google::LogMessage::Flush() at ??:0
@ 0x7f230f624a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f23175440f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f231754d8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f23114f10d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f23114e6954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f231234a6db start_thread at ??:0
@ 0x7f230e12161f clone at ??:0
W20260809 02:12:25.509332 20717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:25.509564 20717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:25.509591 20717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:25.510895 20717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:25.510946 20717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:25.510977 20717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:12:25.512610 20717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:41507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=40259
--tserver_master_addrs=127.5.9.190:45531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:25.512878 20717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:25.513132 20717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:25.518671 20754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:25.518700 20756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:25.518703 20753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:25.519204 20717 server_base.cc:1034] running on GCE node
I20260809 02:12:25.519418 20717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:25.519671 20717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:25.520844 20717 hybrid_clock.cc:648] HybridClock initialized: now 1786241545520829 us; error 31 us; skew 500 ppm
Aug 09 02:12:25 dist-test-slave-p3t5 krb5kdc[19533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241545, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:25.524008 20717 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:12:25.524613 20717 webserver.cc:460] Webserver started at http://127.5.9.129:40259/ using document root <none> and password file <none>
I20260809 02:12:25.524945 20717 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:25.525023 20717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:25.526855 20717 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:25.527781 20764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:25.527995 20717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:25.528079 20717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "7d53e1c10ec94ea3b47bcd94c414bb97"
format_stamp: "Formatted at 2026-08-09 02:12:23 on dist-test-slave-p3t5"
server_key: "49a225c8f0824f3017f1f87fffd8646e"
server_key_iv: "bcf460ef7907bf90aa83774f231494d9"
server_key_version: "encryptionkey@0"
I20260809 02:12:25.528450 20717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:25.544548 20717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:25.547109 20717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:25.547292 20717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:25.547556 20717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:25.548050 20771 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:12:25.550148 20717 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:12:25.550248 20717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 02:12:25.550333 20717 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:12:25.551021 20717 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:12:25.551079 20717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 02:12:25.551153 20771 tablet_bootstrap.cc:492] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97: Bootstrap starting.
I20260809 02:12:25.558631 20717 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:41507
I20260809 02:12:25.558796 20883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:41507 every 8 connection(s)
I20260809 02:12:25.559099 20717 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:12:25.562743 20771 log.cc:824] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:25.563385 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 20717
I20260809 02:12:25.563711 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:33231
--local_ip_for_outbound_sockets=127.5.9.130
--tserver_master_addrs=127.5.9.190:45531
--webserver_port=43817
--webserver_interface=127.5.9.130
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:12:25.565418 20771 tablet_bootstrap.cc:492] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:12:25.565809 20771 tablet_bootstrap.cc:492] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97: Bootstrap complete.
I20260809 02:12:25.566860 20771 ts_tablet_manager.cc:1397] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.006s
I20260809 02:12:25.568019 20771 raft_consensus.cc:348] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } }
I20260809 02:12:25.568301 20771 raft_consensus.cc:729] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d53e1c10ec94ea3b47bcd94c414bb97, State: Initialized, Role: FOLLOWER
I20260809 02:12:25.568461 20771 consensus_queue.cc:260] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } }
I20260809 02:12:25.568780 20771 ts_tablet_manager.cc:1428] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260809 02:12:25.644454 20887 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:12:25.559845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:35101)
0809 02:12:25.560021 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:25.560070 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:25.560551 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:25.644206 (+ 83655us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
W20260809 02:12:25.644753 20884 heartbeater.cc:644] Failed to heartbeat to 127.5.9.190:45531 (0 consecutive failures): Network error: Failed to ping master at 127.5.9.190:45531: Client connection negotiation failed: client connection to 127.5.9.190:45531: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 02:12:25.645093 20887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.644971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:44753)
0809 02:12:25.645027 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:25.645067 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 02:12:25.645457 20887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.645372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:51539)
0809 02:12:25.645417 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:25.645441 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 02:12:25.645778 20884 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:12:25.673403 20889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:25.673691 20889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:25.673739 20889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:25.674998 20889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:25.675071 20889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:25.675118 20889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:12:25.676719 20889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:33231
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=43817
--tserver_master_addrs=127.5.9.190:45531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:25.677024 20889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:25.677277 20889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:25.682801 20895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:25.682803 20896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:25.683148 20898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:25.683131 20889 server_base.cc:1034] running on GCE node
I20260809 02:12:25.683480 20889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:25.683795 20889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:25.684971 20889 hybrid_clock.cc:648] HybridClock initialized: now 1786241545684947 us; error 41 us; skew 500 ppm
Aug 09 02:12:25 dist-test-slave-p3t5 krb5kdc[19533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241545, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:25.687856 20889 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:12:25.688357 20889 webserver.cc:460] Webserver started at http://127.5.9.130:43817/ using document root <none> and password file <none>
I20260809 02:12:25.688627 20889 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:25.688674 20889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:25.690155 20889 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:12:25.690798 20905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:25.691015 20889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:12:25.691115 20889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "3c3e4281cf15481197e7667321267601"
format_stamp: "Formatted at 2026-08-09 02:12:24 on dist-test-slave-p3t5"
server_key: "79b0bc1a2234816f701131b3cc3f5cd4"
server_key_iv: "e7b1ff2d86f8cbd7a08923f9a03c7375"
server_key_version: "encryptionkey@0"
I20260809 02:12:25.691506 20889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:25.702558 20889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:25.704583 20889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:25.704751 20889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:25.705000 20889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:25.705317 20889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:12:25.705369 20889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:25.705440 20889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:12:25.705471 20889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:25.711979 20889 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:33231
I20260809 02:12:25.712036 21020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:33231 every 8 connection(s)
I20260809 02:12:25.712435 20889 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:12:25.713521 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.713131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:45839)
0809 02:12:25.713282 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:25.713365 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
W20260809 02:12:25.713979 21022 heartbeater.cc:644] Failed to heartbeat to 127.5.9.190:45531 (0 consecutive failures): Network error: Failed to ping master at 127.5.9.190:45531: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
I20260809 02:12:25.714279 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.714183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:34897)
0809 02:12:25.714226 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:25.714258 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 02:12:25.714532 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.714482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:41067)
0809 02:12:25.714502 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:25.714521 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260809 02:12:25.714643 21022 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 02:12:25.721690 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 20889
I20260809 02:12:25.721956 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:46001
--local_ip_for_outbound_sockets=127.5.9.131
--tserver_master_addrs=127.5.9.190:45531
--webserver_port=42065
--webserver_interface=127.5.9.131
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:25.807283 21025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:25.807548 21025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:25.807588 21025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:25.808780 21025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:25.808854 21025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:25.808899 21025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:12:25.810458 21025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:46001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=42065
--tserver_master_addrs=127.5.9.190:45531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:25.810747 21025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:25.810995 21025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:25.816362 21031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:25.816372 21033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:25.816362 21030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:25.816792 21025 server_base.cc:1034] running on GCE node
I20260809 02:12:25.817003 21025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:25.817227 21025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:25.818368 21025 hybrid_clock.cc:648] HybridClock initialized: now 1786241545818355 us; error 32 us; skew 500 ppm
Aug 09 02:12:25 dist-test-slave-p3t5 krb5kdc[19533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241545, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:25.821039 21025 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:12:25.821539 21025 webserver.cc:460] Webserver started at http://127.5.9.131:42065/ using document root <none> and password file <none>
I20260809 02:12:25.821815 21025 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:25.821866 21025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:25.823367 21025 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:25.824043 21040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:25.824225 21025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:25.824321 21025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "92ef8f93814a4ec19caf2e766e221413"
format_stamp: "Formatted at 2026-08-09 02:12:24 on dist-test-slave-p3t5"
server_key: "620790ddd217977932153341cafb26bf"
server_key_iv: "0f7bc02ae5d03fe1da31b6e4e10a7c83"
server_key_version: "encryptionkey@0"
I20260809 02:12:25.824720 21025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:25.839071 21025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:25.841297 21025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:25.841491 21025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:25.841807 21025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:25.842155 21025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:12:25.842209 21025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:25.842252 21025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:12:25.842281 21025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:25.848544 21025 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:46001
I20260809 02:12:25.848591 21159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:46001 every 8 connection(s)
I20260809 02:12:25.848978 21025 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:12:25.850214 21163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.849798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:57163)
0809 02:12:25.849982 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:25.850066 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
W20260809 02:12:25.850679 21161 heartbeater.cc:644] Failed to heartbeat to 127.5.9.190:45531 (0 consecutive failures): Network error: Failed to ping master at 127.5.9.190:45531: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
I20260809 02:12:25.851048 21163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.850886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:44305)
0809 02:12:25.850980 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:25.851018 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 02:12:25.851434 21163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:25.851302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:53529)
0809 02:12:25.851373 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:25.851412 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 02:12:25.851610 21161 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 02:12:25.858251 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 21025
I20260809 02:12:26.646458 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:26.646190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:59353)
0809 02:12:26.646344 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:26.646387 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":66,"threads_started":1}
I20260809 02:12:26.715262 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:26.715019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:46971)
0809 02:12:26.715156 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:26.715199 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260809 02:12:26.852272 21167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:26.852019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:57443)
0809 02:12:26.852156 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:26.852204 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":67,"threads_started":1}
I20260809 02:12:26.928185 21168 raft_consensus.cc:482] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:12:26.928326 21168 raft_consensus.cc:504] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } }
I20260809 02:12:26.928764 21168 leader_election.cc:288] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231), 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001)
Aug 09 02:12:26 dist-test-slave-p3t5 krb5kdc[19533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241545, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.130@KRBTEST.COM
Aug 09 02:12:26 dist-test-slave-p3t5 krb5kdc[19533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241545, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.131@KRBTEST.COM
I20260809 02:12:26.940558 21169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:26.929105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:46001 (local address 127.5.9.129:45165)
0809 02:12:26.929232 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:26.929243 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:26.929296 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:26.930221 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:26.930224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:26.930672 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:26.930897 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:26.930900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:26.932360 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:26.932361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:26.933191 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:26.933194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:26.933891 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:26.933892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:26.934009 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:26.934624 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:26.934633 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:26.936830 (+ 2197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:26.938811 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:26.938812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:26.938813 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:26.939131 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:26.939464 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:26.939465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:26.939465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:26.939550 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:26.939978 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:26.939985 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:26.940112 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 02:12:26.940223 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 02:12:26.940441 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":5,"thread_start_us":59,"threads_started":1}
I20260809 02:12:26.940572 21170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:26.929150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:33609 (local address 127.5.9.130:33231)
0809 02:12:26.929314 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:26.929328 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:26.929384 (+ 56us) server_negotiation.cc:407] Connection header received
0809 02:12:26.929961 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:26.929964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:26.930013 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:26.930180 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:26.931026 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:26.932192 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:26.933301 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:26.933790 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:26.933918 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:26.936550 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:26.936555 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:26.936556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:26.936570 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:26.938630 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:26.939196 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:26.939201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:26.939202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:26.939327 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:26.939624 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:26.939625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:26.939626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:26.939894 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:26.939984 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:26.940296 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 02:12:26.940446 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260809 02:12:26.940579 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:26.929235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:45165 (local address 127.5.9.131:46001)
0809 02:12:26.929442 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:26.929450 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:26.929478 (+ 28us) server_negotiation.cc:407] Connection header received
0809 02:12:26.929981 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:26.929983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:26.930022 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:26.930181 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:26.931028 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:26.932193 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:26.933298 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:26.933790 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:26.933907 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:26.936961 (+ 3054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:26.936965 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:26.936965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:26.936976 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:26.938667 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:26.939243 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:26.939245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:26.939245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:26.939349 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:26.939622 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:26.939623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:26.939623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:26.939893 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:26.939985 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:26.940296 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 02:12:26.940438 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":81,"threads_started":1}
I20260809 02:12:26.940656 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:26.928989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:33231 (local address 127.5.9.129:33609)
0809 02:12:26.929062 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:26.929079 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:26.929163 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:26.930221 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:26.930224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:26.930671 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:26.930890 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:26.930897 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:26.932377 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:26.932378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:26.933193 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:26.933195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:26.933889 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:26.933890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:26.934010 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:26.934699 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:26.934704 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:26.936393 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:26.938770 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:26.938781 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:26.938783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:26.939108 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:26.939416 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:26.939417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:26.939418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:26.939551 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:26.939983 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:26.939991 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:26.940113 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 02:12:26.940223 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 02:12:26.940523 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":244}
W20260809 02:12:26.942096 20766 leader_election.cc:341] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001): Not found: Tablet not found: 848724802b54487684e67816ea91fa42
W20260809 02:12:26.942232 20766 leader_election.cc:341] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231): Not found: Tablet not found: 848724802b54487684e67816ea91fa42
I20260809 02:12:26.942265 20766 leader_election.cc:302] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d53e1c10ec94ea3b47bcd94c414bb97; no voters: 3c3e4281cf15481197e7667321267601, 92ef8f93814a4ec19caf2e766e221413
I20260809 02:12:26.942412 21168 raft_consensus.cc:2726] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:12:27.647397 21172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:27.647128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:52641)
0809 02:12:27.647294 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:27.647341 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":71,"threads_started":1}
I20260809 02:12:27.716050 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:27.715800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:41459)
0809 02:12:27.715938 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:27.715989 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260809 02:12:27.853204 21174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:27.852888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:58029)
0809 02:12:27.853097 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:27.853150 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":93,"threads_started":1}
I20260809 02:12:28.648348 21175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:28.648077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:34119)
0809 02:12:28.648240 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:28.648292 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
I20260809 02:12:28.716949 21176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:28.716700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:51015)
0809 02:12:28.716851 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:28.716898 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":66,"threads_started":1}
I20260809 02:12:28.854142 21177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:28.853862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:40523)
0809 02:12:28.854043 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:28.854089 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":67,"threads_started":1}
I20260809 02:12:29.079634 21178 raft_consensus.cc:482] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:12:29.079746 21178 raft_consensus.cc:504] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } }
I20260809 02:12:29.079926 21178 leader_election.cc:288] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231), 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001)
W20260809 02:12:29.080390 20766 leader_election.cc:341] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231): Not found: Tablet not found: 848724802b54487684e67816ea91fa42
W20260809 02:12:29.080468 20766 leader_election.cc:341] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001): Not found: Tablet not found: 848724802b54487684e67816ea91fa42
I20260809 02:12:29.080484 20766 leader_election.cc:302] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d53e1c10ec94ea3b47bcd94c414bb97; no voters: 3c3e4281cf15481197e7667321267601, 92ef8f93814a4ec19caf2e766e221413
I20260809 02:12:29.080560 21178 raft_consensus.cc:2726] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:12:29.649310 21179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:29.649019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:50515)
0809 02:12:29.649190 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:29.649242 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
I20260809 02:12:29.717883 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:29.717534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:51905)
0809 02:12:29.717783 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:29.717830 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":161,"threads_started":1}
I20260809 02:12:29.855146 21181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:29.854856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:34069)
0809 02:12:29.855038 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:29.855091 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":68,"threads_started":1}
I20260809 02:12:30.650281 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:30.650004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:56261)
0809 02:12:30.650175 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:30.650227 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":89,"threads_started":1}
I20260809 02:12:30.718663 21183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:30.718426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:46305)
0809 02:12:30.718563 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:30.718611 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
I20260809 02:12:30.855937 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:30.855658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:37859)
0809 02:12:30.855838 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:30.855883 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":87,"threads_started":1}
I20260809 02:12:31.029670 21185 raft_consensus.cc:482] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:12:31.029793 21185 raft_consensus.cc:504] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } }
I20260809 02:12:31.030050 21185 leader_election.cc:288] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231), 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001)
W20260809 02:12:31.030593 20766 leader_election.cc:341] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001): Not found: Tablet not found: 848724802b54487684e67816ea91fa42
W20260809 02:12:31.030696 20766 leader_election.cc:341] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231): Not found: Tablet not found: 848724802b54487684e67816ea91fa42
I20260809 02:12:31.030731 20766 leader_election.cc:302] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d53e1c10ec94ea3b47bcd94c414bb97; no voters: 3c3e4281cf15481197e7667321267601, 92ef8f93814a4ec19caf2e766e221413
I20260809 02:12:31.030831 21185 raft_consensus.cc:2726] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:12:31.651206 21186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:31.650925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:56345)
0809 02:12:31.651095 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:31.651149 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":109,"threads_started":1}
I20260809 02:12:31.719521 21187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:31.719236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:51791)
0809 02:12:31.719417 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:31.719466 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":85,"threads_started":1}
I20260809 02:12:31.856675 21189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:31.856421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:45281)
0809 02:12:31.856582 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:31.856623 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":81,"threads_started":1}
I20260809 02:12:32.652071 21190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:32.651799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:52891)
0809 02:12:32.651956 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:32.652016 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260809 02:12:32.720413 21191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:32.720160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:52779)
0809 02:12:32.720315 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:32.720360 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260809 02:12:32.857527 21192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:32.857257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:54203)
0809 02:12:32.857431 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:32.857474 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":69,"threads_started":1}
I20260809 02:12:33.652938 21193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:33.652684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:39935)
0809 02:12:33.652847 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:33.652884 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260809 02:12:33.721293 21194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:33.721017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:55529)
0809 02:12:33.721186 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:33.721243 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":100,"threads_started":1}
I20260809 02:12:33.858342 21195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:33.858103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:57331)
0809 02:12:33.858241 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:33.858290 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260809 02:12:34.653923 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:34.653558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:33375)
0809 02:12:34.653787 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:34.653852 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":149,"threads_started":1}
I20260809 02:12:34.722127 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:34.721832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:45923)
0809 02:12:34.722006 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:34.722060 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":109,"threads_started":1}
I20260809 02:12:34.859138 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:34.858906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:46931)
0809 02:12:34.859041 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:34.859086 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260809 02:12:35.373488 21199 raft_consensus.cc:482] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:12:35.373636 21199 raft_consensus.cc:504] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } }
I20260809 02:12:35.373867 21199 leader_election.cc:288] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231), 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001)
W20260809 02:12:35.374366 20766 leader_election.cc:341] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231): Not found: Tablet not found: 848724802b54487684e67816ea91fa42
W20260809 02:12:35.374460 20766 leader_election.cc:341] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001): Not found: Tablet not found: 848724802b54487684e67816ea91fa42
I20260809 02:12:35.374480 20766 leader_election.cc:302] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d53e1c10ec94ea3b47bcd94c414bb97; no voters: 3c3e4281cf15481197e7667321267601, 92ef8f93814a4ec19caf2e766e221413
I20260809 02:12:35.374600 21199 raft_consensus.cc:2726] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:12:35.654942 21200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:35.654577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:44007)
0809 02:12:35.654795 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:35.654885 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":75,"threads_started":1}
I20260809 02:12:35.723011 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:35.722728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:45709)
0809 02:12:35.722878 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:35.722935 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":65,"threads_started":1}
I20260809 02:12:35.859942 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:35.859659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:44723)
0809 02:12:35.859812 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:35.859858 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":67,"threads_started":1}
I20260809 02:12:36.655711 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:36.655468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:45739)
0809 02:12:36.655614 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:36.655658 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":64,"threads_started":1}
I20260809 02:12:36.723943 21204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:36.723680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:42899)
0809 02:12:36.723838 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:36.723888 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
I20260809 02:12:36.860832 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:36.860486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:49613)
0809 02:12:36.860685 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:36.860735 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":84,"threads_started":1}
I20260809 02:12:37.656519 21206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:37.656275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:39615)
0809 02:12:37.656421 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:37.656465 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260809 02:12:37.724978 21207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:37.724640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:49791)
0809 02:12:37.724866 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:37.724920 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":84,"threads_started":1}
I20260809 02:12:37.861764 21208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:37.861457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:59957)
0809 02:12:37.861600 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:37.861709 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":66,"threads_started":1}
I20260809 02:12:37.879575 21209 raft_consensus.cc:482] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:12:37.879685 21209 raft_consensus.cc:504] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3e4281cf15481197e7667321267601" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33231 } } peers { permanent_uuid: "92ef8f93814a4ec19caf2e766e221413" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 46001 } } peers { permanent_uuid: "7d53e1c10ec94ea3b47bcd94c414bb97" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 41507 } }
I20260809 02:12:37.879827 21209 leader_election.cc:288] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231), 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001)
W20260809 02:12:37.880281 20766 leader_election.cc:341] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3c3e4281cf15481197e7667321267601 (127.5.9.130:33231): Not found: Tablet not found: 848724802b54487684e67816ea91fa42
W20260809 02:12:37.880371 20766 leader_election.cc:341] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92ef8f93814a4ec19caf2e766e221413 (127.5.9.131:46001): Not found: Tablet not found: 848724802b54487684e67816ea91fa42
I20260809 02:12:37.880412 20766 leader_election.cc:302] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d53e1c10ec94ea3b47bcd94c414bb97; no voters: 3c3e4281cf15481197e7667321267601, 92ef8f93814a4ec19caf2e766e221413
I20260809 02:12:37.880506 21209 raft_consensus.cc:2726] T 848724802b54487684e67816ea91fa42 P 7d53e1c10ec94ea3b47bcd94c414bb97 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:12:38.657480 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:38.657125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:39475)
0809 02:12:38.657344 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:38.657392 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":87,"threads_started":1}
I20260809 02:12:38.725912 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:38.725600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:46535)
0809 02:12:38.725810 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:38.725859 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":136,"threads_started":1}
I20260809 02:12:38.862586 21212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:38.862311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:35397)
0809 02:12:38.862484 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:38.862534 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":69,"threads_started":1}
I20260809 02:12:39.658357 21213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:39.658098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:51457)
0809 02:12:39.658250 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:39.658299 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
I20260809 02:12:39.726754 21214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:39.726495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:60555)
0809 02:12:39.726635 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:39.726687 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260809 02:12:39.863541 21215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:39.863239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:38415)
0809 02:12:39.863423 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:39.863484 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":66,"threads_started":1}
I20260809 02:12:40.659267 21216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:40.658995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.129:46753)
0809 02:12:40.659146 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:40.659200 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260809 02:12:40.727754 21217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:40.727467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.130:52577)
0809 02:12:40.727662 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:40.727702 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":116,"threads_started":1}
I20260809 02:12:40.864564 21218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:40.864313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:45531 (local address 127.5.9.131:42601)
0809 02:12:40.864462 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:40.864510 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260809 02:12:40.868999 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 20717
I20260809 02:12:40.877318 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 20889
I20260809 02:12:40.883978 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 21025
2026-08-09T02:12:40Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18600 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:12:40 dist-test-slave-p3t5 krb5kdc[21221](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:12:40 dist-test-slave-p3t5 krb5kdc[21221](info): set up 2 sockets
Aug 09 02:12:40 dist-test-slave-p3t5 krb5kdc[21221](info): commencing operation
krb5kdc: starting...
W20260809 02:12:42.057109 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:12:42 dist-test-slave-p3t5 krb5kdc[21221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:12:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:12:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:12:42.192435 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:43039
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:43039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:42.279145 21234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:42.279405 21234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:42.279448 21234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:42.280666 21234 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 02:12:42.280746 21234 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 02:12:42.280769 21234 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 02:12:42.280789 21234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:12:42.280820 21234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:42.280849 21234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:42.280876 21234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:12:42.280900 21234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:12:42.282604 21234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:43039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:43039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:42.282927 21234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:42.283198 21234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:42.288525 21239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:42.288525 21243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:42.288563 21240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:42.289059 21234 server_base.cc:1034] running on GCE node
I20260809 02:12:42.289309 21234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:42.289588 21234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:42.290800 21234 hybrid_clock.cc:648] HybridClock initialized: now 1786241562290773 us; error 43 us; skew 500 ppm
Aug 09 02:12:42 dist-test-slave-p3t5 krb5kdc[21221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241562, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:42.293674 21234 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:12:42.294250 21234 webserver.cc:460] Webserver started at http://127.5.9.190:43637/ using document root <none> and password file <none>
I20260809 02:12:42.294540 21234 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:42.294591 21234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:42.294762 21234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:12:42.295958 21234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "c0bb51080df2466fab6e88e4f202abf9"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "8ea43e9306e6cd2038ae5d57faa49072"
server_key_iv: "c1027700b005f436ee8a822b323aa0ca"
server_key_version: "encryptionkey@0"
I20260809 02:12:42.296365 21234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "c0bb51080df2466fab6e88e4f202abf9"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "8ea43e9306e6cd2038ae5d57faa49072"
server_key_iv: "c1027700b005f436ee8a822b323aa0ca"
server_key_version: "encryptionkey@0"
I20260809 02:12:42.298013 21234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:12:42.298870 21250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:42.299080 21234 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:42.299181 21234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "c0bb51080df2466fab6e88e4f202abf9"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "8ea43e9306e6cd2038ae5d57faa49072"
server_key_iv: "c1027700b005f436ee8a822b323aa0ca"
server_key_version: "encryptionkey@0"
I20260809 02:12:42.299268 21234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:42.322336 21234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:42.324585 21234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:42.324751 21234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:42.329329 21234 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:43039
I20260809 02:12:42.329370 21307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:43039 every 8 connection(s)
I20260809 02:12:42.329807 21234 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:12:42.330842 21308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:42.333319 21308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Bootstrap starting.
I20260809 02:12:42.333976 21308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:42.334250 21308 log.cc:824] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:42.335062 21308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: No bootstrap required, opened a new log
I20260809 02:12:42.336709 21308 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } }
I20260809 02:12:42.336879 21308 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:42.336903 21308 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0bb51080df2466fab6e88e4f202abf9, State: Initialized, Role: FOLLOWER
I20260809 02:12:42.337082 21308 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } }
I20260809 02:12:42.337175 21308 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:12:42.337210 21308 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:12:42.337244 21308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:42.337857 21308 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } }
I20260809 02:12:42.337954 21308 leader_election.cc:302] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0bb51080df2466fab6e88e4f202abf9; no voters:
I20260809 02:12:42.338109 21308 leader_election.cc:288] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:12:42.338223 21311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:12:42.338397 21308 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:12:42.338441 21311 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 1 LEADER]: Becoming Leader. State: Replica: c0bb51080df2466fab6e88e4f202abf9, State: Running, Role: LEADER
I20260809 02:12:42.338532 21311 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } }
I20260809 02:12:42.339089 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 21234
I20260809 02:12:42.339221 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:12:42.339345 21313 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0bb51080df2466fab6e88e4f202abf9. Latest consensus state: current_term: 1 leader_uuid: "c0bb51080df2466fab6e88e4f202abf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } } }
I20260809 02:12:42.339419 21313 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [sys.catalog]: This master's current role is: LEADER
I20260809 02:12:42.339520 5158 external_mini_cluster.cc:1428] Setting key a48e14b92ccce70a1284777dd08eba58
I20260809 02:12:42.339836 21320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:12:42.339984 21312 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0bb51080df2466fab6e88e4f202abf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } } }
I20260809 02:12:42.340078 21312 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [sys.catalog]: This master's current role is: LEADER
I20260809 02:12:42.340433 21320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:12:42.342080 21320 catalog_manager.cc:1347] Generated new cluster ID: d634e3913d2d4449a13e7abb4b2e879a
I20260809 02:12:42.342136 21320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:12:42 dist-test-slave-p3t5 krb5kdc[21221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:42.350526 21329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:42.340214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42252 (local address 127.5.9.190:43039)
0809 02:12:42.340471 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:42.340483 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:42.340521 (+ 38us) server_negotiation.cc:407] Connection header received
0809 02:12:42.340642 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:42.340649 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:42.340695 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:42.340851 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:42.341840 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.342905 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.343723 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.344165 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.344280 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:42.346877 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:42.346883 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:42.346883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:42.346897 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:42.348734 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:42.349143 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:42.349152 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:42.349152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:42.349256 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:42.349475 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:42.349476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:42.349476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:42.349818 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:42.349996 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:42.350100 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 02:12:42.350278 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":98,"threads_started":1}
I20260809 02:12:42.355427 21320 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:12:42.356174 21320 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:12:42.361487 21320 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Generated new TSK 0
I20260809 02:12:42.361747 21320 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:12:42.401499 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:43039
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:42.504508 21334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:42.504737 21334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:42.504763 21334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:42.506088 21334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:42.506137 21334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:42.506165 21334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:12:42.507771 21334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:43039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:42.508044 21334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:42.508252 21334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:42.513697 21343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:42.513706 21340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:42.513695 21339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:42.513769 21334 server_base.cc:1034] running on GCE node
I20260809 02:12:42.514097 21334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:42.514308 21334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:42.515486 21334 hybrid_clock.cc:648] HybridClock initialized: now 1786241562515470 us; error 29 us; skew 500 ppm
Aug 09 02:12:42 dist-test-slave-p3t5 krb5kdc[21221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241562, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:42.518337 21334 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:12:42.518842 21334 webserver.cc:460] Webserver started at http://127.5.9.129:35753/ using document root <none> and password file <none>
I20260809 02:12:42.519038 21334 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:42.519078 21334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:42.519170 21334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:12:42.520259 21334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "9abc6e65e7dd48c7a4447cdccd69e49c"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "3dd8110918d3014d6b4fff3a459a882c"
server_key_iv: "64691f4b1c002450aa156cb495ac8255"
server_key_version: "encryptionkey@0"
I20260809 02:12:42.520584 21334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "9abc6e65e7dd48c7a4447cdccd69e49c"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "3dd8110918d3014d6b4fff3a459a882c"
server_key_iv: "64691f4b1c002450aa156cb495ac8255"
server_key_version: "encryptionkey@0"
I20260809 02:12:42.521989 21334 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:12:42.522722 21350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:42.522926 21334 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:12:42.523017 21334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "9abc6e65e7dd48c7a4447cdccd69e49c"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "3dd8110918d3014d6b4fff3a459a882c"
server_key_iv: "64691f4b1c002450aa156cb495ac8255"
server_key_version: "encryptionkey@0"
I20260809 02:12:42.523077 21334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:42.537343 21334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:42.539556 21334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:42.539741 21334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:42.540016 21334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:42.540357 21334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:12:42.540393 21334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:42.540417 21334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:12:42.540477 21334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:42.546813 21334 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:33625
I20260809 02:12:42.546898 21469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:33625 every 8 connection(s)
I20260809 02:12:42.547264 21334 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:12:42.548635 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 21334
I20260809 02:12:42.548765 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:12:42.548987 5158 external_mini_cluster.cc:1428] Setting key 17f23b2332f92b674165d5106fb0a206
Aug 09 02:12:42 dist-test-slave-p3t5 krb5kdc[21221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241562, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
I20260809 02:12:42.559661 21329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:42.548160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:57685 (local address 127.5.9.190:43039)
0809 02:12:42.548255 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:42.548257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:42.548445 (+ 188us) server_negotiation.cc:407] Connection header received
0809 02:12:42.549237 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:42.549239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:42.549257 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:42.549314 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:42.550742 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.551325 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.552550 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.552909 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.552994 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:42.555478 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:42.555481 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:42.555481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:42.555492 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:42.557366 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:42.558004 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:42.558006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:42.558006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:42.558128 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:42.558492 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:42.558493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:42.558494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:42.558750 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:42.558947 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:42.559321 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 02:12:42.559479 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 02:12:42.559746 21472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:42.548107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:57685)
0809 02:12:42.548275 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:42.548315 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:42.549093 (+ 778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:42.549405 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:42.549413 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:42.550314 (+ 901us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:42.550628 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:42.550635 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.551429 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.551430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:42.552440 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:42.552442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.553023 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.553024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:42.553129 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:42.553798 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:42.553808 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:42.555366 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:42.557492 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:42.557503 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:42.557505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:42.557922 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:42.558225 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:42.558226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:42.558226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:42.558392 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:42.558898 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:42.558906 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:42.559109 (+ 203us) client_negotiation.cc:769] Sending connection context
0809 02:12:42.559231 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 02:12:42.559440 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260809 02:12:42.560690 21470 heartbeater.cc:347] Connected to a master server at 127.5.9.190:43039
I20260809 02:12:42.560818 21470 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:42.561059 21470 heartbeater.cc:511] Master 127.5.9.190:43039 requested a full tablet report, sending...
I20260809 02:12:42.561748 21270 ts_manager.cc:194] Registered new tserver with Master: 9abc6e65e7dd48c7a4447cdccd69e49c (127.5.9.129:33625)
I20260809 02:12:42.562711 21270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:57685
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:12:42.580499 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:43039
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:42.666752 21475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:42.666975 21475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:42.667002 21475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:42.668174 21475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:42.668215 21475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:42.668241 21475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:12:42.669710 21475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:43039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:42.669927 21475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:42.670111 21475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:42.675587 21483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:42.675572 21481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:42.675712 21475 server_base.cc:1034] running on GCE node
W20260809 02:12:42.675568 21480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:42.676044 21475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:42.676357 21475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:42.677544 21475 hybrid_clock.cc:648] HybridClock initialized: now 1786241562677516 us; error 44 us; skew 500 ppm
Aug 09 02:12:42 dist-test-slave-p3t5 krb5kdc[21221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241562, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:42.680361 21475 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:12:42.680855 21475 webserver.cc:460] Webserver started at http://127.5.9.130:35547/ using document root <none> and password file <none>
I20260809 02:12:42.681061 21475 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:42.681103 21475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:42.681201 21475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:12:42.682374 21475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "83b551ca410f45fdb9c82efbae122086"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "d95b63bec94b61eb062527c8e6b524ff"
server_key_iv: "edcf6a88c568493fa891b3557652a353"
server_key_version: "encryptionkey@0"
I20260809 02:12:42.682704 21475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "83b551ca410f45fdb9c82efbae122086"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "d95b63bec94b61eb062527c8e6b524ff"
server_key_iv: "edcf6a88c568493fa891b3557652a353"
server_key_version: "encryptionkey@0"
I20260809 02:12:42.684078 21475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:12:42.684813 21490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:42.685075 21475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:42.685160 21475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "83b551ca410f45fdb9c82efbae122086"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "d95b63bec94b61eb062527c8e6b524ff"
server_key_iv: "edcf6a88c568493fa891b3557652a353"
server_key_version: "encryptionkey@0"
I20260809 02:12:42.685225 21475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:42.698190 21475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:42.700302 21475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:42.700425 21475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:42.700635 21475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:42.701086 21475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:12:42.701124 21475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:42.701175 21475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:12:42.701202 21475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:42.707530 21475 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:43693
I20260809 02:12:42.707610 21607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:43693 every 8 connection(s)
I20260809 02:12:42.707974 21475 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:12:42 dist-test-slave-p3t5 krb5kdc[21221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241562, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:42.716800 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 21475
I20260809 02:12:42.716930 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:12:42.717166 5158 external_mini_cluster.cc:1428] Setting key f3714994e3614bc12c0f0de2cc9f0ed5
I20260809 02:12:42.718290 21329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:42.708820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:60503 (local address 127.5.9.190:43039)
0809 02:12:42.708885 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:42.708888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:42.709091 (+ 203us) server_negotiation.cc:407] Connection header received
0809 02:12:42.709683 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:42.709685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:42.709698 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:42.709749 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:42.710747 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.711396 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.712427 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.712643 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.712733 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:42.715022 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:42.715023 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:42.715023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:42.715030 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:42.716545 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:42.717022 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:42.717024 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:42.717024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:42.717088 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:42.717396 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:42.717397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:42.717397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:42.717572 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:42.717697 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:42.718044 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 02:12:42.718155 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 02:12:42.718358 21610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:42.708801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:60503)
0809 02:12:42.708970 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:42.709009 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:42.709542 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:42.709827 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:42.709831 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:42.710429 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:42.710637 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:42.710642 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.711500 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.711501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:42.712326 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:42.712329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.712746 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.712747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:42.712873 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:42.713416 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:42.713424 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:42.714923 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:42.716637 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:42.716643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:42.716645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:42.716887 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:42.717150 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:42.717150 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:42.717151 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:42.717305 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:42.717687 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:42.717693 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:42.717882 (+ 189us) client_negotiation.cc:769] Sending connection context
0809 02:12:42.718008 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 02:12:42.718163 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260809 02:12:42.719014 21608 heartbeater.cc:347] Connected to a master server at 127.5.9.190:43039
I20260809 02:12:42.719147 21608 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:42.719331 21608 heartbeater.cc:511] Master 127.5.9.190:43039 requested a full tablet report, sending...
I20260809 02:12:42.719774 21270 ts_manager.cc:194] Registered new tserver with Master: 83b551ca410f45fdb9c82efbae122086 (127.5.9.130:43693)
I20260809 02:12:42.720261 21270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:60503
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:12:42.746088 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:43039
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:12:42.832252 21614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:12:42.832520 21614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:12:42.832559 21614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:12:42.833755 21614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:12:42.833834 21614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:12:42.833881 21614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:12:42.835427 21614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:43039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:12:42.835731 21614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:12:42.835999 21614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:12:42.841708 21622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:42.841707 21619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:12:42.841763 21620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:12:42.841953 21614 server_base.cc:1034] running on GCE node
I20260809 02:12:42.842161 21614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:12:42.842391 21614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:12:42.843541 21614 hybrid_clock.cc:648] HybridClock initialized: now 1786241562843505 us; error 57 us; skew 500 ppm
Aug 09 02:12:42 dist-test-slave-p3t5 krb5kdc[21221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241562, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:12:42.846313 21614 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:12:42.846796 21614 webserver.cc:460] Webserver started at http://127.5.9.131:42981/ using document root <none> and password file <none>
I20260809 02:12:42.847075 21614 fs_manager.cc:359] Metadata directory not provided
I20260809 02:12:42.847126 21614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:12:42.847309 21614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:12:42.848453 21614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "398b3a4c775d4bdfa1cd79af9e4b603c"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "c74d7d423f4908d71ac7f716737162d6"
server_key_iv: "42af483b6652352c8115af7d64e04f9d"
server_key_version: "encryptionkey@0"
I20260809 02:12:42.848829 21614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "398b3a4c775d4bdfa1cd79af9e4b603c"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "c74d7d423f4908d71ac7f716737162d6"
server_key_iv: "42af483b6652352c8115af7d64e04f9d"
server_key_version: "encryptionkey@0"
I20260809 02:12:42.850411 21614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:12:42.851287 21630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:42.851589 21614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:12:42.851708 21614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "398b3a4c775d4bdfa1cd79af9e4b603c"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "c74d7d423f4908d71ac7f716737162d6"
server_key_iv: "42af483b6652352c8115af7d64e04f9d"
server_key_version: "encryptionkey@0"
I20260809 02:12:42.851816 21614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:12:42.863157 21614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:12:42.865340 21614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:12:42.865504 21614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:12:42.865787 21614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:12:42.866127 21614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:12:42.866161 21614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:42.866185 21614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:12:42.866199 21614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:12:42.872388 21614 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:41333
I20260809 02:12:42.872439 21748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:41333 every 8 connection(s)
I20260809 02:12:42.872766 21614 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
Aug 09 02:12:42 dist-test-slave-p3t5 krb5kdc[21221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241562, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:12:42.882125 21329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:42.873705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:39395 (local address 127.5.9.190:43039)
0809 02:12:42.873773 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:42.873775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:42.873999 (+ 224us) server_negotiation.cc:407] Connection header received
0809 02:12:42.874479 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:42.874480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:42.874491 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:42.874552 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:42.875496 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.875990 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.876962 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.877148 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.877204 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:42.879475 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:42.879477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:42.879477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:42.879484 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:42.880772 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:42.881181 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:42.881182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:42.881183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:42.881239 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:42.881468 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:42.881469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:42.881469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:42.881579 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:42.881692 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:42.881984 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 02:12:42.882054 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 02:12:42.882297 21751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:42.873665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:39395)
0809 02:12:42.873845 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:42.873899 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:42.874391 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:42.874606 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:42.874609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:42.875198 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:12:42.875410 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:42.875415 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.876075 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.876075 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:42.876870 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:42.876871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.877224 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.877225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:42.877326 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:42.877869 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:12:42.877892 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:12:42.879377 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:12:42.880853 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:42.880860 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:42.880862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:42.881113 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:42.881301 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:12:42.881301 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:12:42.881302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:12:42.881407 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:12:42.881716 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:12:42.881721 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:12:42.881844 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 02:12:42.881950 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 02:12:42.882086 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260809 02:12:42.882416 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 21614
I20260809 02:12:42.882511 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:12:42.882769 5158 external_mini_cluster.cc:1428] Setting key ed675768156322fd30eddd3c595b48fc
I20260809 02:12:42.882825 21749 heartbeater.cc:347] Connected to a master server at 127.5.9.190:43039
I20260809 02:12:42.882951 21749 heartbeater.cc:464] Registering TS with master...
I20260809 02:12:42.883154 21749 heartbeater.cc:511] Master 127.5.9.190:43039 requested a full tablet report, sending...
I20260809 02:12:42.883553 21270 ts_manager.cc:194] Registered new tserver with Master: 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333)
I20260809 02:12:42.883865 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:12:42.884006 21270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:39395
I20260809 02:12:42.891373 21329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:42.885374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42262 (local address 127.5.9.190:43039)
0809 02:12:42.885421 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:42.885423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:42.885529 (+ 106us) server_negotiation.cc:407] Connection header received
0809 02:12:42.885573 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:42.885574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:42.885582 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:42.885679 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:12:42.886216 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.886676 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.887364 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.887549 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.887599 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:42.888806 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:12:42.888808 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:12:42.888808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:12:42.888814 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:12:42.890402 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:42.890726 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:42.890727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:42.890727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:42.890779 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:12:42.890931 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:12:42.890932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:12:42.890932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:12:42.891061 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:12:42.891135 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:42.891225 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 02:12:42.891284 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:12:42.892988 21270 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42262:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:12:42.894714 21270 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:12:42.901312 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:42.897159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:33625 (local address 127.0.0.1:35930)
0809 02:12:42.897499 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:42.897544 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:42.897664 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:42.897981 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:42.897984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:42.897986 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:42.898063 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:42.898065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.899044 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.899045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:42.900121 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:42.900123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.900990 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.900991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:42.901085 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:42.901100 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 02:12:42.901176 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:12:42.901224 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"spinlock_wait_cycles":768,"thread_start_us":88,"threads_started":1}
I20260809 02:12:42.901355 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:42.897294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35930 (local address 127.5.9.129:33625)
0809 02:12:42.897516 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:42.897522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:42.897764 (+ 242us) server_negotiation.cc:407] Connection header received
0809 02:12:42.897818 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:42.897821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:42.897839 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:42.897927 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:42.898172 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.898931 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.900207 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.900880 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.900969 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:42.901031 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:42.901211 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 02:12:42.901260 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":110,"threads_started":1}
I20260809 02:12:42.901592 21761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:42.897159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:43693 (local address 127.0.0.1:58604)
0809 02:12:42.897675 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:42.897685 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:42.897739 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:42.898114 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:42.898115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:42.898117 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:42.898178 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:42.898180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.899051 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.899053 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:42.900339 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:42.900342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.901344 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.901345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:42.901427 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:42.901432 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:12:42.901486 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 02:12:42.901523 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":57,"thread_start_us":55,"threads_started":1}
I20260809 02:12:42.902292 21400 tablet_service.cc:1467] Processing CreateTablet for tablet ce71f1b135364e9bbb073223b4bbd3fe (DEFAULT_TABLE table=test-table [id=761882ec12424126a8b04fba015f4e8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:12:42.902716 21400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce71f1b135364e9bbb073223b4bbd3fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:42.903025 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:42.897382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:41333 (local address 127.0.0.1:36760)
0809 02:12:42.898038 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:42.898048 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:42.898104 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:42.899205 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:42.899205 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:42.899208 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:42.899292 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:42.899294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.900666 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.900667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:42.902014 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:42.902017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.902784 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.902786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:42.902845 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:42.902849 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:12:42.902909 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 02:12:42.902958 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":39,"thread_start_us":78,"threads_started":1}
I20260809 02:12:42.903070 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:42.897489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36760 (local address 127.5.9.131:41333)
0809 02:12:42.897694 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:42.897699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:42.898108 (+ 409us) server_negotiation.cc:407] Connection header received
0809 02:12:42.899092 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:42.899095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:42.899114 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:42.899192 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:42.899373 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.900573 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.902098 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.902681 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.902768 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:42.902823 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:42.902948 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 02:12:42.902986 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":94,"threads_started":1}
I20260809 02:12:42.903635 21680 tablet_service.cc:1467] Processing CreateTablet for tablet ce71f1b135364e9bbb073223b4bbd3fe (DEFAULT_TABLE table=test-table [id=761882ec12424126a8b04fba015f4e8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:12:42.903934 21680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce71f1b135364e9bbb073223b4bbd3fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:42.905985 21766 tablet_bootstrap.cc:492] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: Bootstrap starting.
I20260809 02:12:42.906116 21763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:42.897295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58604 (local address 127.5.9.130:43693)
0809 02:12:42.897601 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:42.897605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:42.897744 (+ 139us) server_negotiation.cc:407] Connection header received
0809 02:12:42.897911 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:42.897913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:42.897930 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:42.898020 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:42.898255 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.898936 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.900451 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:42.901181 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:42.901251 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:42.901307 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:42.901726 (+ 419us) server_negotiation.cc:299] Negotiation successful
0809 02:12:42.901780 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":99,"threads_started":1}
I20260809 02:12:42.906867 21767 tablet_bootstrap.cc:492] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c: Bootstrap starting.
I20260809 02:12:42.906839 21540 tablet_service.cc:1467] Processing CreateTablet for tablet ce71f1b135364e9bbb073223b4bbd3fe (DEFAULT_TABLE table=test-table [id=761882ec12424126a8b04fba015f4e8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:12:42.906911 21766 tablet_bootstrap.cc:654] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:42.907194 21540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce71f1b135364e9bbb073223b4bbd3fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:12:42.907613 21767 tablet_bootstrap.cc:654] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:42.907660 21766 log.cc:824] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:42.908244 21767 log.cc:824] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:42.908576 21766 tablet_bootstrap.cc:492] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: No bootstrap required, opened a new log
I20260809 02:12:42.908663 21766 ts_tablet_manager.cc:1397] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:12:42.909214 21767 tablet_bootstrap.cc:492] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c: No bootstrap required, opened a new log
I20260809 02:12:42.909300 21767 ts_tablet_manager.cc:1397] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:12:42.910326 21770 tablet_bootstrap.cc:492] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086: Bootstrap starting.
I20260809 02:12:42.910725 21766 raft_consensus.cc:348] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } }
I20260809 02:12:42.910912 21766 raft_consensus.cc:374] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:42.910967 21766 raft_consensus.cc:729] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9abc6e65e7dd48c7a4447cdccd69e49c, State: Initialized, Role: FOLLOWER
I20260809 02:12:42.911060 21770 tablet_bootstrap.cc:654] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086: Neither blocks nor log segments found. Creating new log.
I20260809 02:12:42.911087 21766 consensus_queue.cc:260] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } }
I20260809 02:12:42.911379 21766 ts_tablet_manager.cc:1428] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:12:42.911531 21470 heartbeater.cc:503] Master 127.5.9.190:43039 was elected leader, sending a full tablet report...
I20260809 02:12:42.911477 21767 raft_consensus.cc:348] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } }
I20260809 02:12:42.911685 21767 raft_consensus.cc:374] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:42.911721 21767 raft_consensus.cc:729] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 398b3a4c775d4bdfa1cd79af9e4b603c, State: Initialized, Role: FOLLOWER
I20260809 02:12:42.911831 21767 consensus_queue.cc:260] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } }
I20260809 02:12:42.912135 21767 ts_tablet_manager.cc:1428] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:12:42.912130 21770 log.cc:824] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086: Log is configured to *not* fsync() on all Append() calls
I20260809 02:12:42.912160 21749 heartbeater.cc:503] Master 127.5.9.190:43039 was elected leader, sending a full tablet report...
I20260809 02:12:42.913034 21770 tablet_bootstrap.cc:492] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086: No bootstrap required, opened a new log
I20260809 02:12:42.913120 21770 ts_tablet_manager.cc:1397] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:12:42.914558 21770 raft_consensus.cc:348] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } }
I20260809 02:12:42.914693 21770 raft_consensus.cc:374] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:12:42.914716 21770 raft_consensus.cc:729] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b551ca410f45fdb9c82efbae122086, State: Initialized, Role: FOLLOWER
I20260809 02:12:42.914862 21770 consensus_queue.cc:260] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } }
I20260809 02:12:42.915148 21770 ts_tablet_manager.cc:1428] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 02:12:42.915177 21608 heartbeater.cc:503] Master 127.5.9.190:43039 was elected leader, sending a full tablet report...
W20260809 02:12:42.958577 21609 tablet.cc:2369] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:12:43.048207 21471 tablet.cc:2369] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:12:43.095647 21774 raft_consensus.cc:482] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:12:43.095798 21774 raft_consensus.cc:504] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } }
I20260809 02:12:43.096136 21774 leader_election.cc:288] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9abc6e65e7dd48c7a4447cdccd69e49c (127.5.9.129:33625), 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333)
I20260809 02:12:43.099421 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:43.096680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:42829 (local address 127.5.9.131:41333)
0809 02:12:43.096728 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:43.096730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:43.096831 (+ 101us) server_negotiation.cc:407] Connection header received
0809 02:12:43.096869 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:43.096870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:43.096883 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:43.096957 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:43.097157 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:43.097661 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:43.098656 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:43.099035 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:43.099124 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:43.099134 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:43.099287 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 02:12:43.099324 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:12:43.099422 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:43.096516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:54143 (local address 127.5.9.129:33625)
0809 02:12:43.096578 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:43.096580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:43.096595 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:12:43.096715 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:43.096717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:43.096732 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:43.096857 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:12:43.097077 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:43.097594 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:43.098657 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:43.099034 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:43.099123 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:43.099134 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:43.099306 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 02:12:43.099344 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 02:12:43.099442 21610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:43.096378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:33625 (local address 127.5.9.130:54143)
0809 02:12:43.096473 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:43.096488 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:43.096597 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:43.096868 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:43.096869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:43.096871 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:43.096979 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:43.096982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:43.097726 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:43.097727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:43.098552 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:43.098555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:43.099133 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:43.099134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:43.099220 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:43.099223 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:12:43.099300 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 02:12:43.099345 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":19}
I20260809 02:12:43.099449 21775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:43.096509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:41333 (local address 127.5.9.130:42829)
0809 02:12:43.096740 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:43.096751 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:12:43.096817 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:12:43.096969 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:12:43.096970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:12:43.096971 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:12:43.097075 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:43.097077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:43.097757 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:43.097758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:43.098552 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:12:43.098554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:12:43.099125 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:12:43.099126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:12:43.099207 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:43.099210 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:12:43.099279 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 02:12:43.099324 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":2,"thread_start_us":129,"threads_started":1}
I20260809 02:12:43.099901 21422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce71f1b135364e9bbb073223b4bbd3fe" candidate_uuid: "83b551ca410f45fdb9c82efbae122086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" is_pre_election: true
I20260809 02:12:43.099956 21700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce71f1b135364e9bbb073223b4bbd3fe" candidate_uuid: "83b551ca410f45fdb9c82efbae122086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" is_pre_election: true
I20260809 02:12:43.100107 21700 raft_consensus.cc:2454] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83b551ca410f45fdb9c82efbae122086 in term 0.
I20260809 02:12:43.100097 21422 raft_consensus.cc:2454] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83b551ca410f45fdb9c82efbae122086 in term 0.
I20260809 02:12:43.100353 21494 leader_election.cc:302] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 398b3a4c775d4bdfa1cd79af9e4b603c, 83b551ca410f45fdb9c82efbae122086; no voters:
I20260809 02:12:43.100538 21774 raft_consensus.cc:2781] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:12:43.100603 21774 raft_consensus.cc:482] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:12:43.100621 21774 raft_consensus.cc:3037] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:43.101388 21774 raft_consensus.cc:504] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } }
I20260809 02:12:43.101518 21774 leader_election.cc:288] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [CANDIDATE]: Term 1 election: Requested vote from peers 9abc6e65e7dd48c7a4447cdccd69e49c (127.5.9.129:33625), 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333)
I20260809 02:12:43.101784 21422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce71f1b135364e9bbb073223b4bbd3fe" candidate_uuid: "83b551ca410f45fdb9c82efbae122086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c"
I20260809 02:12:43.101887 21422 raft_consensus.cc:3037] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:43.101866 21700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce71f1b135364e9bbb073223b4bbd3fe" candidate_uuid: "83b551ca410f45fdb9c82efbae122086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c"
I20260809 02:12:43.102007 21700 raft_consensus.cc:3037] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:12:43.102756 21422 raft_consensus.cc:2454] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83b551ca410f45fdb9c82efbae122086 in term 1.
I20260809 02:12:43.102990 21700 raft_consensus.cc:2454] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83b551ca410f45fdb9c82efbae122086 in term 1.
I20260809 02:12:43.103178 21494 leader_election.cc:302] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 398b3a4c775d4bdfa1cd79af9e4b603c, 83b551ca410f45fdb9c82efbae122086; no voters:
I20260809 02:12:43.103288 21774 raft_consensus.cc:2781] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:12:43.103464 21774 raft_consensus.cc:686] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [term 1 LEADER]: Becoming Leader. State: Replica: 83b551ca410f45fdb9c82efbae122086, State: Running, Role: LEADER
I20260809 02:12:43.103590 21774 consensus_queue.cc:237] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } }
I20260809 02:12:43.104511 21267 catalog_manager.cc:5495] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83b551ca410f45fdb9c82efbae122086 (127.5.9.130). New cstate: current_term: 1 leader_uuid: "83b551ca410f45fdb9c82efbae122086" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } health_report { overall_health: HEALTHY } } }
W20260809 02:12:43.123377 21750 tablet.cc:2369] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:12:43.129004 21763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:43.126651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58606 (local address 127.5.9.130:43693)
0809 02:12:43.126720 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 02:12:43.126722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:12:43.126759 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:12:43.126819 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:12:43.126820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:12:43.126848 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:12:43.126945 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:12:43.127101 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:43.127589 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:43.128324 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:12:43.128513 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:12:43.128565 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:12:43.128708 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:12:43.128815 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:12:43.128876 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 02:12:43.128918 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 02:12:43.128936 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:12:43.131107 21422 raft_consensus.cc:1261] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Refusing update from remote peer 83b551ca410f45fdb9c82efbae122086: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:12:43.131107 21700 raft_consensus.cc:1261] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c [term 1 FOLLOWER]: Refusing update from remote peer 83b551ca410f45fdb9c82efbae122086: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:12:43.131525 21776 consensus_queue.cc:1035] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:12:43.131677 21774 consensus_queue.cc:1035] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:12:43.134524 21781 mvcc.cc:205] Tried to move back new op lower bound from 7316445442581762048 to 7316445442472779776. Current Snapshot: MvccSnapshot[applied={T|T < 7316445442581762048}]
I20260809 02:12:43.136837 21269 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:12:43.136966 21269 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:12:43.137818 21269 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Sending DeleteTablet for 3 replicas of tablet ce71f1b135364e9bbb073223b4bbd3fe
I20260809 02:12:43.138211 21400 tablet_service.cc:1514] Processing DeleteTablet for tablet ce71f1b135364e9bbb073223b4bbd3fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:12:43 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:35930
I20260809 02:12:43.138226 21540 tablet_service.cc:1514] Processing DeleteTablet for tablet ce71f1b135364e9bbb073223b4bbd3fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:12:43 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:58604
I20260809 02:12:43.138463 21790 tablet_replica.cc:330] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086: stopping tablet replica
I20260809 02:12:43.138495 21680 tablet_service.cc:1514] Processing DeleteTablet for tablet ce71f1b135364e9bbb073223b4bbd3fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:12:43 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:36760
I20260809 02:12:43.138576 21790 raft_consensus.cc:2229] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:12:43.138712 21790 raft_consensus.cc:2258] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:12:43.138732 21792 tablet_replica.cc:330] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c: stopping tablet replica
I20260809 02:12:43.138876 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 21334
I20260809 02:12:43.138878 21792 raft_consensus.cc:2229] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:12:43.139036 21792 raft_consensus.cc:2258] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:12:43.139361 21790 ts_tablet_manager.cc:1905] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:12:43.139493 21792 ts_tablet_manager.cc:1905] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:12:43.141280 21792 ts_tablet_manager.cc:1918] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:12:43.141341 21792 log.cc:1196] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/ce71f1b135364e9bbb073223b4bbd3fe
I20260809 02:12:43.141589 21792 ts_tablet_manager.cc:1939] T ce71f1b135364e9bbb073223b4bbd3fe P 398b3a4c775d4bdfa1cd79af9e4b603c: Deleting consensus metadata
I20260809 02:12:43.141731 21790 ts_tablet_manager.cc:1918] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:12:43.141814 21790 log.cc:1196] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/ce71f1b135364e9bbb073223b4bbd3fe
I20260809 02:12:43.142186 21790 ts_tablet_manager.cc:1939] T ce71f1b135364e9bbb073223b4bbd3fe P 83b551ca410f45fdb9c82efbae122086: Deleting consensus metadata
I20260809 02:12:43.142742 21253 catalog_manager.cc:4841] TS 83b551ca410f45fdb9c82efbae122086 (127.5.9.130:43693): tablet ce71f1b135364e9bbb073223b4bbd3fe (table test-table [id=761882ec12424126a8b04fba015f4e8d]) successfully deleted
I20260809 02:12:43.146456 21254 catalog_manager.cc:4841] TS 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333): tablet ce71f1b135364e9bbb073223b4bbd3fe (table test-table [id=761882ec12424126a8b04fba015f4e8d]) successfully deleted
W20260809 02:12:43.152109 21254 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:33625 (error 108)
I20260809 02:12:43.152275 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 21475
I20260809 02:12:43.153604 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:12:43.152644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:33625 (local address 127.0.0.1:35938)
0809 02:12:43.153504 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0809 02:12:43.153552 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:33625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":836}
W20260809 02:12:43.154127 21254 catalog_manager.cc:4584] TS 9abc6e65e7dd48c7a4447cdccd69e49c (127.5.9.129:33625): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ce71f1b135364e9bbb073223b4bbd3fe: Network error: Client connection negotiation failed: client connection to 127.5.9.129:33625: connect: Connection refused (error 111)
I20260809 02:12:43.158403 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 21614
I20260809 02:12:43.163882 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 21234
I20260809 02:13:03.170857 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:43039
--webserver_interface=127.5.9.190
--webserver_port=43637
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:43039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:03.256875 21794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:03.257117 21794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:03.257143 21794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:03.258275 21794 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 02:13:03.258311 21794 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 02:13:03.258325 21794 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 02:13:03.258337 21794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:13:03.258350 21794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:03.258363 21794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:03.258375 21794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:13:03.258387 21794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:13:03.259835 21794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:43039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:43039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=43637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:03.260051 21794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:03.260224 21794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:03.270498 21801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:03.270512 21800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:03.270498 21804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:03.270536 21794 server_base.cc:1034] running on GCE node
I20260809 02:13:03.271160 21794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:03.271524 21794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:03.272693 21794 hybrid_clock.cc:648] HybridClock initialized: now 1786241583272683 us; error 48 us; skew 500 ppm
Aug 09 02:13:03 dist-test-slave-p3t5 krb5kdc[21221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241583, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:03.280501 21794 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:13:03.281158 21794 webserver.cc:460] Webserver started at http://127.5.9.190:43637/ using document root <none> and password file <none>
I20260809 02:13:03.281442 21794 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:03.281536 21794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:03.283182 21794 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:03.284001 21812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:03.284252 21794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:03.284381 21794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "c0bb51080df2466fab6e88e4f202abf9"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "8ea43e9306e6cd2038ae5d57faa49072"
server_key_iv: "c1027700b005f436ee8a822b323aa0ca"
server_key_version: "encryptionkey@0"
I20260809 02:13:03.284929 21794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:03.293489 21794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:03.296042 21794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:03.296221 21794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:03.300696 21794 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:43039
I20260809 02:13:03.300734 21867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:43039 every 8 connection(s)
I20260809 02:13:03.301177 21794 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:13:03.302834 21868 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:13:03.303480 21868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Bootstrap starting.
I20260809 02:13:03.307575 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 21794
I20260809 02:13:03.307920 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:33625
--local_ip_for_outbound_sockets=127.5.9.129
--tserver_master_addrs=127.5.9.190:43039
--webserver_port=35753
--webserver_interface=127.5.9.129
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:13:03.312566 21868 log.cc:824] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:03.314857 21868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:13:03.315197 21868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Bootstrap complete.
I20260809 02:13:03.317804 21868 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } }
I20260809 02:13:03.318109 21868 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0bb51080df2466fab6e88e4f202abf9, State: Initialized, Role: FOLLOWER
I20260809 02:13:03.318313 21868 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } }
I20260809 02:13:03.318399 21868 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:13:03.318460 21868 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:13:03.318536 21868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:13:03.319667 21868 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } }
I20260809 02:13:03.319803 21868 leader_election.cc:302] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0bb51080df2466fab6e88e4f202abf9; no voters:
I20260809 02:13:03.320062 21868 leader_election.cc:288] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 02:13:03.320106 21872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:13:03.320237 21872 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 2 LEADER]: Becoming Leader. State: Replica: c0bb51080df2466fab6e88e4f202abf9, State: Running, Role: LEADER
I20260809 02:13:03.320381 21868 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:13:03.320400 21872 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } }
I20260809 02:13:03.320817 21873 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c0bb51080df2466fab6e88e4f202abf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } } }
I20260809 02:13:03.320916 21873 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:03.321167 21874 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0bb51080df2466fab6e88e4f202abf9. Latest consensus state: current_term: 2 leader_uuid: "c0bb51080df2466fab6e88e4f202abf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } } }
I20260809 02:13:03.321254 21874 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:03.321461 21879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:13:03.322438 21879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:13:03.322556 21879 catalog_manager.cc:1259] Loaded cluster ID: d634e3913d2d4449a13e7abb4b2e879a
I20260809 02:13:03.322640 21879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:13:03.323570 21879 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:13:03.323779 21879 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Loaded TSK: 0
I20260809 02:13:03.332572 21879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Generated new TSK 1
I20260809 02:13:03.332701 21879 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:13:03.406082 21870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:03.406443 21870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:03.406490 21870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:03.407886 21870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:03.407965 21870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:03.408013 21870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:13:03.409754 21870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:33625
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=35753
--tserver_master_addrs=127.5.9.190:43039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:03.410058 21870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:03.410333 21870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:03.415923 21896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:03.416075 21897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:03.415920 21899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:03.416558 21870 server_base.cc:1034] running on GCE node
I20260809 02:13:03.416821 21870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:03.417124 21870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:03.418324 21870 hybrid_clock.cc:648] HybridClock initialized: now 1786241583418306 us; error 36 us; skew 500 ppm
Aug 09 02:13:03 dist-test-slave-p3t5 krb5kdc[21221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241583, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:03.421752 21870 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:13:03.422328 21870 webserver.cc:460] Webserver started at http://127.5.9.129:35753/ using document root <none> and password file <none>
I20260809 02:13:03.422593 21870 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:03.422686 21870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:03.424356 21870 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:13:03.425215 21906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:03.425482 21870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:03.425587 21870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "9abc6e65e7dd48c7a4447cdccd69e49c"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "3dd8110918d3014d6b4fff3a459a882c"
server_key_iv: "64691f4b1c002450aa156cb495ac8255"
server_key_version: "encryptionkey@0"
I20260809 02:13:03.426116 21870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:03.447901 21870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:03.450726 21870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:03.450914 21870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:03.451187 21870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:03.451709 21913 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:13:03.453264 21870 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:13:03.453333 21870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 02:13:03.453372 21870 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:13:03.454257 21870 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:13:03.454339 21870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:03.454391 21913 tablet_bootstrap.cc:492] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: Bootstrap starting.
I20260809 02:13:03.461257 21870 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:33625
I20260809 02:13:03.461484 22032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:33625 every 8 connection(s)
I20260809 02:13:03.461796 21870 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:13:03.464630 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 21870
I20260809 02:13:03.464994 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:43693
--local_ip_for_outbound_sockets=127.5.9.130
--tserver_master_addrs=127.5.9.190:43039
--webserver_port=35547
--webserver_interface=127.5.9.130
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:13:03.466137 21913 log.cc:824] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:03.470374 21913 tablet_bootstrap.cc:492] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:13:03.470825 21913 tablet_bootstrap.cc:492] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: Bootstrap complete.
I20260809 02:13:03.472257 21913 ts_tablet_manager.cc:1397] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.009s
I20260809 02:13:03.474015 21913 raft_consensus.cc:348] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } }
I20260809 02:13:03.475103 21913 raft_consensus.cc:729] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9abc6e65e7dd48c7a4447cdccd69e49c, State: Initialized, Role: FOLLOWER
I20260809 02:13:03.475371 21913 consensus_queue.cc:260] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } }
I20260809 02:13:03.475795 21913 ts_tablet_manager.cc:1428] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
Aug 09 02:13:03 dist-test-slave-p3t5 krb5kdc[21221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241583, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:13:03.488349 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.462691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:38919 (local address 127.5.9.190:43039)
0809 02:13:03.466564 (+ 3873us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:03.474510 (+ 7946us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:03.474573 (+ 63us) server_negotiation.cc:407] Connection header received
0809 02:13:03.474740 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:03.474748 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:03.474815 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:03.475007 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:03.477291 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.478831 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.479992 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.480571 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.480675 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.483886 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:03.483892 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:03.483892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:03.483909 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:03.486318 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:03.486777 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:03.486781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:03.486782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:03.486914 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:03.487278 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:03.487279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:03.487279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:03.487546 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:03.487645 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:03.488010 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 02:13:03.488166 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3812,"thread_start_us":91,"threads_started":1}
I20260809 02:13:03.488953 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.462561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:38919)
0809 02:13:03.462757 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:03.462803 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:03.463537 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:03.476448 (+ 12911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:03.476452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:03.476800 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:03.477141 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:03.477148 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.478962 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.478963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:03.479853 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:03.479857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.480781 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.480782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:03.480924 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.481670 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:13:03.481682 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:03.483732 (+ 2050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:03.486399 (+ 2667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:03.486407 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:03.486409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:03.486653 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:03.487009 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:03.487010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:03.487010 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:03.487211 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:03.487648 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:03.487657 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:03.487819 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 02:13:03.488565 (+ 746us) client_negotiation.cc:240] Negotiation successful
0809 02:13:03.488755 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":119,"threads_started":1}
I20260809 02:13:03.489760 22033 heartbeater.cc:347] Connected to a master server at 127.5.9.190:43039
I20260809 02:13:03.489869 22033 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:03.490057 22033 heartbeater.cc:511] Master 127.5.9.190:43039 requested a full tablet report, sending...
I20260809 02:13:03.490885 21830 ts_manager.cc:194] Registered new tserver with Master: 9abc6e65e7dd48c7a4447cdccd69e49c (127.5.9.129:33625)
I20260809 02:13:03.492085 21830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:38919
I20260809 02:13:03.493191 22033 heartbeater.cc:503] Master 127.5.9.190:43039 was elected leader, sending a full tablet report...
I20260809 02:13:03.497954 22040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.492555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:33625 (local address 127.0.0.1:44828)
0809 02:13:03.493339 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:03.493369 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:03.493478 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:03.493805 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:03.493809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:03.493811 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:03.493889 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:03.493891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.495113 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.495114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:03.496524 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:03.496528 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.497553 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.497554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:03.497679 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.497696 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 02:13:03.497773 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 02:13:03.497861 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":108,"threads_started":1}
I20260809 02:13:03.498392 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.492834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44828 (local address 127.5.9.129:33625)
0809 02:13:03.493016 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:03.493021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:03.493456 (+ 435us) server_negotiation.cc:407] Connection header received
0809 02:13:03.493558 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:03.493561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:03.493577 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:03.493740 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:03.493979 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.495009 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.496640 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.497450 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.497540 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.497603 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:03.498257 (+ 654us) server_negotiation.cc:299] Negotiation successful
0809 02:13:03.498321 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
I20260809 02:13:03.499117 21965 tablet_service.cc:1514] Processing DeleteTablet for tablet ce71f1b135364e9bbb073223b4bbd3fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:12:43 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:44828
I20260809 02:13:03.499801 22042 tablet_replica.cc:330] stopping tablet replica
I20260809 02:13:03.499900 22042 raft_consensus.cc:2229] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:13:03.499953 22042 raft_consensus.cc:2258] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:13:03.500293 22042 ts_tablet_manager.cc:1905] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:13:03.502863 22042 ts_tablet_manager.cc:1918] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:13:03.502959 22042 log.cc:1196] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal/wals/ce71f1b135364e9bbb073223b4bbd3fe
I20260809 02:13:03.503216 22042 ts_tablet_manager.cc:1939] T ce71f1b135364e9bbb073223b4bbd3fe P 9abc6e65e7dd48c7a4447cdccd69e49c: Deleting consensus metadata
I20260809 02:13:03.503780 21816 catalog_manager.cc:4841] TS 9abc6e65e7dd48c7a4447cdccd69e49c (127.5.9.129:33625): tablet ce71f1b135364e9bbb073223b4bbd3fe (table test-table [id=761882ec12424126a8b04fba015f4e8d]) successfully deleted
W20260809 02:13:03.580307 22038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:03.580531 22038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:03.580556 22038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:03.581738 22038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:03.581784 22038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:03.581811 22038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:13:03.583452 22038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:43693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=35547
--tserver_master_addrs=127.5.9.190:43039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:03.583688 22038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:03.583881 22038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:03.589277 22047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:03.589282 22050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:03.589406 22038 server_base.cc:1034] running on GCE node
W20260809 02:13:03.589484 22048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:03.589812 22038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:03.590077 22038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:03.591245 22038 hybrid_clock.cc:648] HybridClock initialized: now 1786241583591238 us; error 42 us; skew 500 ppm
Aug 09 02:13:03 dist-test-slave-p3t5 krb5kdc[21221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241583, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:03.593962 22038 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:13:03.594520 22038 webserver.cc:460] Webserver started at http://127.5.9.130:35547/ using document root <none> and password file <none>
I20260809 02:13:03.594765 22038 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:03.594849 22038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:03.596297 22038 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:03.596980 22058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:03.597178 22038 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:03.597283 22038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "83b551ca410f45fdb9c82efbae122086"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "d95b63bec94b61eb062527c8e6b524ff"
server_key_iv: "edcf6a88c568493fa891b3557652a353"
server_key_version: "encryptionkey@0"
I20260809 02:13:03.597715 22038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:03.619503 22038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:03.622371 22038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:03.622509 22038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:03.622778 22038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:03.623133 22038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:03.623171 22038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:03.623217 22038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:03.623243 22038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:03.629174 22038 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:43693
I20260809 02:13:03.629252 22177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:43693 every 8 connection(s)
I20260809 02:13:03.629694 22038 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:13:03.633881 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 22038
I20260809 02:13:03.634217 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:41333
--local_ip_for_outbound_sockets=127.5.9.131
--tserver_master_addrs=127.5.9.190:43039
--webserver_port=42981
--webserver_interface=127.5.9.131
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:13:03 dist-test-slave-p3t5 krb5kdc[21221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241583, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:13:03.640959 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.630494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:48791 (local address 127.5.9.190:43039)
0809 02:13:03.630568 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:03.630570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:03.630805 (+ 235us) server_negotiation.cc:407] Connection header received
0809 02:13:03.631365 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:03.631366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:03.631391 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:03.631457 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:03.632429 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.632935 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.634179 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.634525 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.634596 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.637519 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:03.637521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:03.637522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:03.637531 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:03.639272 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:03.639762 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:03.639764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:03.639764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:03.639846 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:03.640133 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:03.640134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:03.640135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:03.640326 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:03.640417 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:03.640739 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 02:13:03.640848 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:13:03.641167 22180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.630476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:48791)
0809 02:13:03.630666 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:03.630711 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:03.631256 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:03.631537 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:03.631541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:03.632140 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:03.632337 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:03.632342 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.633052 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.633053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:03.634079 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:03.634085 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.634625 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.634625 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:03.634736 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.635358 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:13:03.635367 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:03.637363 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:03.639386 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:03.639396 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:03.639398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:03.639662 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:03.639922 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:03.639923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:03.639924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:03.640050 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:03.640429 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:03.640437 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:03.640598 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 02:13:03.640716 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 02:13:03.640900 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":108,"threads_started":1}
I20260809 02:13:03.641798 22178 heartbeater.cc:347] Connected to a master server at 127.5.9.190:43039
I20260809 02:13:03.641908 22178 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:03.642119 22178 heartbeater.cc:511] Master 127.5.9.190:43039 requested a full tablet report, sending...
I20260809 02:13:03.642652 21830 ts_manager.cc:194] Registered new tserver with Master: 83b551ca410f45fdb9c82efbae122086 (127.5.9.130:43693)
I20260809 02:13:03.643136 21830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:48791
W20260809 02:13:03.722955 22181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:03.723186 22181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:03.723212 22181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:03.724462 22181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:03.724514 22181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:03.724542 22181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:13:03.726152 22181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:41333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=42981
--tserver_master_addrs=127.5.9.190:43039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:03.726420 22181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:03.726675 22181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:03.732203 22189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:03.732203 22187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:03.732249 22186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:03.732455 22181 server_base.cc:1034] running on GCE node
I20260809 02:13:03.732661 22181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:03.732895 22181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:03.734078 22181 hybrid_clock.cc:648] HybridClock initialized: now 1786241583734056 us; error 42 us; skew 500 ppm
Aug 09 02:13:03 dist-test-slave-p3t5 krb5kdc[21221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241583, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:03.736778 22181 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:13:03.737298 22181 webserver.cc:460] Webserver started at http://127.5.9.131:42981/ using document root <none> and password file <none>
I20260809 02:13:03.737526 22181 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:03.737583 22181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:03.739048 22181 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:13:03.739638 22197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:03.739846 22181 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:13:03.739946 22181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "398b3a4c775d4bdfa1cd79af9e4b603c"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "c74d7d423f4908d71ac7f716737162d6"
server_key_iv: "42af483b6652352c8115af7d64e04f9d"
server_key_version: "encryptionkey@0"
I20260809 02:13:03.740338 22181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:03.763101 22181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:03.765720 22181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:03.765913 22181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:03.766189 22181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:03.766527 22181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:03.766585 22181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:03.766630 22181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:03.766660 22181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:03.772935 22181 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:41333
I20260809 02:13:03.772981 22323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:41333 every 8 connection(s)
I20260809 02:13:03.773371 22181 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
Aug 09 02:13:03 dist-test-slave-p3t5 krb5kdc[21221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241583, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:13:03.781843 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 22181
I20260809 02:13:03.783902 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.774291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:33863 (local address 127.5.9.190:43039)
0809 02:13:03.774382 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:03.774385 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:03.774563 (+ 178us) server_negotiation.cc:407] Connection header received
0809 02:13:03.775080 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:03.775081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:03.775094 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:03.775155 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:03.776155 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.776661 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.777881 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.778105 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.778174 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.780540 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:03.780542 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:03.780542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:03.780550 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:03.782101 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:03.782706 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:03.782707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:03.782707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:03.782784 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:03.783100 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:03.783100 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:03.783101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:03.783219 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:03.783274 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:03.783650 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 02:13:03.783738 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:13:03.784204 22326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.774223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:33863)
0809 02:13:03.774424 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:03.774462 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:03.774982 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:03.775210 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:03.775213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:03.775820 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:03.776042 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:03.776048 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.776769 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.776770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:03.777775 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:03.777778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.778192 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.778193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:03.778297 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.778956 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:13:03.778965 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:03.780440 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:03.782209 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:03.782217 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:03.782220 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:03.782633 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:03.782857 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:03.782858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:03.782858 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:03.783026 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:03.783291 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:03.783298 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:03.783501 (+ 203us) client_negotiation.cc:769] Sending connection context
0809 02:13:03.783618 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 02:13:03.783799 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":112,"threads_started":1}
I20260809 02:13:03.784719 22324 heartbeater.cc:347] Connected to a master server at 127.5.9.190:43039
I20260809 02:13:03.784821 22324 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:03.785043 22324 heartbeater.cc:511] Master 127.5.9.190:43039 requested a full tablet report, sending...
I20260809 02:13:03.785646 21830 ts_manager.cc:194] Registered new tserver with Master: 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333)
I20260809 02:13:03.786141 21830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:33863
I20260809 02:13:03.789888 22328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.782370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34534 (local address 127.5.9.190:43039)
0809 02:13:03.782495 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:03.782497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:03.782611 (+ 114us) server_negotiation.cc:407] Connection header received
0809 02:13:03.782701 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:03.782702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:03.782715 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:03.782792 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:03.783580 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.784393 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.785409 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.785678 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.785749 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.787136 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:03.787137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:03.787138 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:03.787144 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:03.788556 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:03.789021 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:03.789022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:03.789022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:03.789081 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:03.789262 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:03.789262 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:03.789263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:03.789375 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:03.789506 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:03.789730 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 02:13:03.789808 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260809 02:13:03.790971 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:13:03.793932 22328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.791373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34538 (local address 127.5.9.190:43039)
0809 02:13:03.791421 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:03.791423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:03.791552 (+ 129us) server_negotiation.cc:407] Connection header received
0809 02:13:03.791632 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:03.791633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:03.791666 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:03.791727 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:13:03.791843 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.792356 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.793171 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.793365 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.793419 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.793567 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:13:03.793669 (+ 102us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 02:13:03.793862 (+ 193us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34538: token expired
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 02:13:03.794020 22328 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34538: token expired
W20260809 02:13:03.794245 21754 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.9.190:43039)
I20260809 02:13:03.800410 22328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.794527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34548 (local address 127.5.9.190:43039)
0809 02:13:03.794585 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:03.794587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:03.794595 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:13:03.794650 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:03.794650 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:03.794658 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:03.794697 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:03.795197 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.795640 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.796360 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.796542 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.796598 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.798019 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:03.798020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:03.798020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:03.798026 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:03.799287 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:03.799636 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:03.799636 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:03.799637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:03.799693 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:03.799866 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:03.799866 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:03.799866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:03.800028 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:03.800111 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:03.800226 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 02:13:03.800291 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:13:03.801846 21830 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:13:03.803494 21830 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:13:03.805939 21965 tablet_service.cc:1467] Processing CreateTablet for tablet b0ee32a24b1741bb8cc408062f3edee3 (DEFAULT_TABLE table=test-table [id=6adc73fc7de4499695e18cda9260f694]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:03.806178 21965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0ee32a24b1741bb8cc408062f3edee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:03.807662 21913 tablet_bootstrap.cc:492] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c: Bootstrap starting.
I20260809 02:13:03.808037 21913 tablet_bootstrap.cc:654] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:03.808696 21913 tablet_bootstrap.cc:492] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c: No bootstrap required, opened a new log
I20260809 02:13:03.808760 21913 ts_tablet_manager.cc:1397] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:03.808873 21913 raft_consensus.cc:348] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } }
I20260809 02:13:03.808954 21913 raft_consensus.cc:374] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:03.809011 21913 raft_consensus.cc:729] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9abc6e65e7dd48c7a4447cdccd69e49c, State: Initialized, Role: FOLLOWER
I20260809 02:13:03.809058 21913 consensus_queue.cc:260] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } }
I20260809 02:13:03.809149 21913 ts_tablet_manager.cc:1428] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:13:03.810479 22330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.805536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:43693 (local address 127.0.0.1:59784)
0809 02:13:03.805742 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:03.805753 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:03.805802 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:03.806537 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:03.806539 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:03.806541 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:03.806588 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:03.806589 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.808035 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.808036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:03.809097 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:03.809101 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.810223 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.810224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:03.810278 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.810282 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:13:03.810349 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:13:03.810398 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":24,"thread_start_us":120,"threads_started":1}
I20260809 02:13:03.810495 22332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.806145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59784 (local address 127.5.9.130:43693)
0809 02:13:03.806375 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:03.806380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:03.806390 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:13:03.806432 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:03.806434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:03.806450 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:03.806542 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:03.806648 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.807940 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.809256 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.810064 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.810131 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.810193 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:03.810368 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 02:13:03.810407 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":104,"threads_started":1}
I20260809 02:13:03.810770 22040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.805536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:41333 (local address 127.0.0.1:53046)
0809 02:13:03.805599 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:03.805640 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:03.805693 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:03.806413 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:03.806414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:03.806416 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:03.806477 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:03.806479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.808257 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.808258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:03.809423 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:03.809427 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.810503 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.810504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:03.810573 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.810577 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:13:03.810643 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 02:13:03.810692 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 02:13:03.810782 22331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.805794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53046 (local address 127.5.9.131:41333)
0809 02:13:03.806034 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:03.806040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:03.806105 (+ 65us) server_negotiation.cc:407] Connection header received
0809 02:13:03.806156 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:03.806159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:03.806179 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:03.806635 (+ 456us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:03.806757 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.808035 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.809526 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.810392 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.810500 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.810560 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:03.810658 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 02:13:03.810699 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":165,"threads_started":1}
I20260809 02:13:03.811539 22108 tablet_service.cc:1467] Processing CreateTablet for tablet b0ee32a24b1741bb8cc408062f3edee3 (DEFAULT_TABLE table=test-table [id=6adc73fc7de4499695e18cda9260f694]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:03.811895 22108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0ee32a24b1741bb8cc408062f3edee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:03.814065 22254 tablet_service.cc:1467] Processing CreateTablet for tablet b0ee32a24b1741bb8cc408062f3edee3 (DEFAULT_TABLE table=test-table [id=6adc73fc7de4499695e18cda9260f694]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:03.814399 22254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0ee32a24b1741bb8cc408062f3edee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:03.815060 22334 tablet_bootstrap.cc:492] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086: Bootstrap starting.
I20260809 02:13:03.815853 22334 tablet_bootstrap.cc:654] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:03.816557 22334 log.cc:824] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:03.817469 22334 tablet_bootstrap.cc:492] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086: No bootstrap required, opened a new log
I20260809 02:13:03.817552 22334 ts_tablet_manager.cc:1397] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:13:03.818416 22336 tablet_bootstrap.cc:492] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c: Bootstrap starting.
I20260809 02:13:03.819240 22336 tablet_bootstrap.cc:654] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:03.819804 22336 log.cc:824] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:03.819686 22334 raft_consensus.cc:348] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } }
I20260809 02:13:03.819933 22334 raft_consensus.cc:374] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:03.819996 22334 raft_consensus.cc:729] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b551ca410f45fdb9c82efbae122086, State: Initialized, Role: FOLLOWER
I20260809 02:13:03.820152 22334 consensus_queue.cc:260] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } }
I20260809 02:13:03.820503 22334 ts_tablet_manager.cc:1428] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:13:03.820670 22178 heartbeater.cc:503] Master 127.5.9.190:43039 was elected leader, sending a full tablet report...
I20260809 02:13:03.820770 22336 tablet_bootstrap.cc:492] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c: No bootstrap required, opened a new log
I20260809 02:13:03.820844 22336 ts_tablet_manager.cc:1397] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:13:03.822705 22336 raft_consensus.cc:348] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } }
I20260809 02:13:03.822960 22336 raft_consensus.cc:374] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:03.823016 22336 raft_consensus.cc:729] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 398b3a4c775d4bdfa1cd79af9e4b603c, State: Initialized, Role: FOLLOWER
I20260809 02:13:03.823143 22336 consensus_queue.cc:260] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } }
I20260809 02:13:03.823416 22336 ts_tablet_manager.cc:1428] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:13:03.823485 22324 heartbeater.cc:503] Master 127.5.9.190:43039 was elected leader, sending a full tablet report...
I20260809 02:13:03.879299 22039 raft_consensus.cc:482] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:13:03.879472 22039 raft_consensus.cc:504] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } }
I20260809 02:13:03.879808 22039 leader_election.cc:288] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83b551ca410f45fdb9c82efbae122086 (127.5.9.130:43693), 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333)
W20260809 02:13:03.880512 22179 tablet.cc:2369] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:13:03.883303 22331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.880191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:48345 (local address 127.5.9.131:41333)
0809 02:13:03.880272 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:03.880274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:03.880286 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:13:03.880344 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:03.880345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:03.880359 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:03.880421 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:03.880643 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.881317 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.882450 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.882982 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.883038 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.883047 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:03.883206 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 02:13:03.883246 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:13:03.883328 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.880028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:41333 (local address 127.5.9.129:48345)
0809 02:13:03.880099 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:03.880112 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:03.880179 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:03.880483 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:03.880484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:03.880486 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:03.880548 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:03.880550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.881412 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.881413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:03.882351 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:03.882355 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.883060 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.883061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:03.883125 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.883129 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:13:03.883191 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 02:13:03.883231 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 02:13:03.883659 22340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.880028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:43693 (local address 127.5.9.129:50247)
0809 02:13:03.880251 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:03.880260 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:03.880316 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:03.880510 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:03.880510 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:03.880512 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:03.880987 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:03.880990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.881787 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.881788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:03.882694 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:03.882697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.883387 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.883388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:03.883469 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.883474 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:13:03.883529 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 02:13:03.883566 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":22,"thread_start_us":132,"threads_started":1}
I20260809 02:13:03.883692 22332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.880149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:50247 (local address 127.5.9.130:43693)
0809 02:13:03.880207 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:03.880209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:03.880327 (+ 118us) server_negotiation.cc:407] Connection header received
0809 02:13:03.880389 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:03.880390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:03.880404 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:03.880451 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:03.881079 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.881669 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.882806 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.883299 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.883366 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.883377 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:03.883576 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 02:13:03.883621 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 02:13:03.883738 22275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0ee32a24b1741bb8cc408062f3edee3" candidate_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" is_pre_election: true
I20260809 02:13:03.883916 22275 raft_consensus.cc:2454] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9abc6e65e7dd48c7a4447cdccd69e49c in term 0.
I20260809 02:13:03.884109 22130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0ee32a24b1741bb8cc408062f3edee3" candidate_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83b551ca410f45fdb9c82efbae122086" is_pre_election: true
I20260809 02:13:03.884186 21910 leader_election.cc:302] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 398b3a4c775d4bdfa1cd79af9e4b603c, 9abc6e65e7dd48c7a4447cdccd69e49c; no voters:
I20260809 02:13:03.884243 22130 raft_consensus.cc:2454] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9abc6e65e7dd48c7a4447cdccd69e49c in term 0.
I20260809 02:13:03.884342 22039 raft_consensus.cc:2781] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:13:03.884399 22039 raft_consensus.cc:482] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:13:03.884424 22039 raft_consensus.cc:3037] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:03.885242 22039 raft_consensus.cc:504] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } }
I20260809 02:13:03.885399 22039 leader_election.cc:288] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 1 election: Requested vote from peers 83b551ca410f45fdb9c82efbae122086 (127.5.9.130:43693), 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333)
I20260809 02:13:03.885594 22275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0ee32a24b1741bb8cc408062f3edee3" candidate_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c"
I20260809 02:13:03.885690 22130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0ee32a24b1741bb8cc408062f3edee3" candidate_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83b551ca410f45fdb9c82efbae122086"
I20260809 02:13:03.885771 22130 raft_consensus.cc:3037] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:03.885775 22275 raft_consensus.cc:3037] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:03.886534 22275 raft_consensus.cc:2454] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9abc6e65e7dd48c7a4447cdccd69e49c in term 1.
I20260809 02:13:03.886548 22130 raft_consensus.cc:2454] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9abc6e65e7dd48c7a4447cdccd69e49c in term 1.
I20260809 02:13:03.886759 21910 leader_election.cc:302] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 398b3a4c775d4bdfa1cd79af9e4b603c, 9abc6e65e7dd48c7a4447cdccd69e49c; no voters:
I20260809 02:13:03.886852 22039 raft_consensus.cc:2781] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:03.887022 22039 raft_consensus.cc:686] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 LEADER]: Becoming Leader. State: Replica: 9abc6e65e7dd48c7a4447cdccd69e49c, State: Running, Role: LEADER
I20260809 02:13:03.887120 22039 consensus_queue.cc:237] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } }
I20260809 02:13:03.888082 21830 catalog_manager.cc:5495] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c reported cstate change: term changed from 0 to 1, leader changed from <none> to 9abc6e65e7dd48c7a4447cdccd69e49c (127.5.9.129). New cstate: current_term: 1 leader_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } health_report { overall_health: HEALTHY } } }
I20260809 02:13:03.904174 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.901726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44830 (local address 127.5.9.129:33625)
0809 02:13:03.901799 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:03.901801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:03.901810 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:13:03.901858 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:03.901859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:03.901882 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:03.901972 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:13:03.902105 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.902586 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.903343 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:03.903545 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:03.903623 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:03.903805 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:13:03.903915 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:13:03.903999 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:03.904051 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 02:13:03.904074 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:13:03.905951 22130 raft_consensus.cc:1261] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086 [term 1 FOLLOWER]: Refusing update from remote peer 9abc6e65e7dd48c7a4447cdccd69e49c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:13:03.905944 22275 raft_consensus.cc:1261] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c [term 1 FOLLOWER]: Refusing update from remote peer 9abc6e65e7dd48c7a4447cdccd69e49c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:13:03.906356 22341 consensus_queue.cc:1035] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [LEADER]: Connected to new peer: Peer: permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:13:03.906553 22039 consensus_queue.cc:1035] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [LEADER]: Connected to new peer: Peer: permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:13:03.909032 22344 mvcc.cc:205] Tried to move back new op lower bound from 7316445527675813888 to 7316445527602397184. Current Snapshot: MvccSnapshot[applied={T|T < 7316445527675813888}]
I20260809 02:13:03.910277 22347 mvcc.cc:205] Tried to move back new op lower bound from 7316445527675813888 to 7316445527602397184. Current Snapshot: MvccSnapshot[applied={T|T < 7316445527675813888}]
I20260809 02:13:03.912124 21830 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:13:03.912223 21830 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:13:03.913015 21830 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Sending DeleteTablet for 3 replicas of tablet b0ee32a24b1741bb8cc408062f3edee3
I20260809 02:13:03.913368 21965 tablet_service.cc:1514] Processing DeleteTablet for tablet b0ee32a24b1741bb8cc408062f3edee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:03 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:44828
I20260809 02:13:03.913343 22108 tablet_service.cc:1514] Processing DeleteTablet for tablet b0ee32a24b1741bb8cc408062f3edee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:03 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:59784
I20260809 02:13:03.913381 22254 tablet_service.cc:1514] Processing DeleteTablet for tablet b0ee32a24b1741bb8cc408062f3edee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:03 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:53046
I20260809 02:13:03.913470 22042 tablet_replica.cc:330] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c: stopping tablet replica
I20260809 02:13:03.913515 22042 raft_consensus.cc:2229] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:13:03.913529 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 21870
I20260809 02:13:03.913600 22042 raft_consensus.cc:2258] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:13:03.914989 22353 tablet_replica.cc:330] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086: stopping tablet replica
I20260809 02:13:03.915120 22353 raft_consensus.cc:2229] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:13:03.915239 22353 raft_consensus.cc:2258] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:13:03.915777 22353 ts_tablet_manager.cc:1905] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:13:03.917675 22353 ts_tablet_manager.cc:1918] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:13:03.917774 22353 log.cc:1196] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/b0ee32a24b1741bb8cc408062f3edee3
I20260809 02:13:03.918119 22353 ts_tablet_manager.cc:1939] T b0ee32a24b1741bb8cc408062f3edee3 P 83b551ca410f45fdb9c82efbae122086: Deleting consensus metadata
I20260809 02:13:03.918581 21815 catalog_manager.cc:4841] TS 83b551ca410f45fdb9c82efbae122086 (127.5.9.130:43693): tablet b0ee32a24b1741bb8cc408062f3edee3 (table test-table [id=6adc73fc7de4499695e18cda9260f694]) successfully deleted
W20260809 02:13:03.921411 21816 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:33625 (error 108)
I20260809 02:13:03.921794 22355 tablet_replica.cc:330] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c: stopping tablet replica
I20260809 02:13:03.921952 22355 raft_consensus.cc:2229] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:13:03.922036 22040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:03.921846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:33625 (local address 127.0.0.1:44842)
0809 02:13:03.921932 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:03.921981 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:33625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 02:13:03.922104 22355 raft_consensus.cc:2258] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 02:13:03.922188 21816 catalog_manager.cc:4584] TS 9abc6e65e7dd48c7a4447cdccd69e49c (127.5.9.129:33625): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b0ee32a24b1741bb8cc408062f3edee3: Network error: Client connection negotiation failed: client connection to 127.5.9.129:33625: connect: Connection refused (error 111)
I20260809 02:13:03.922631 22355 ts_tablet_manager.cc:1905] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:13:03.922806 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 22038
I20260809 02:13:03.924644 22355 ts_tablet_manager.cc:1918] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:13:03.924744 22355 log.cc:1196] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/b0ee32a24b1741bb8cc408062f3edee3
I20260809 02:13:03.925036 22355 ts_tablet_manager.cc:1939] T b0ee32a24b1741bb8cc408062f3edee3 P 398b3a4c775d4bdfa1cd79af9e4b603c: Deleting consensus metadata
I20260809 02:13:03.925508 21816 catalog_manager.cc:4841] TS 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333): tablet b0ee32a24b1741bb8cc408062f3edee3 (table test-table [id=6adc73fc7de4499695e18cda9260f694]) successfully deleted
I20260809 02:13:03.930816 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 22181
I20260809 02:13:03.936909 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 21794
I20260809 02:13:23.944365 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:43039
--webserver_interface=127.5.9.190
--webserver_port=43637
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:43039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:24.032065 22357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:24.032294 22357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:24.032320 22357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:24.033566 22357 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 02:13:24.033633 22357 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 02:13:24.033648 22357 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 02:13:24.033660 22357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:13:24.033704 22357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:24.033744 22357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:24.033759 22357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:13:24.033810 22357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:13:24.035410 22357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:43039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:43039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=43637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:24.035633 22357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:24.035841 22357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:24.041512 22364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:24.041589 22357 server_base.cc:1034] running on GCE node
W20260809 02:13:24.041534 22366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:24.041536 22362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:24.042217 22357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:24.042573 22357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:24.043767 22357 hybrid_clock.cc:648] HybridClock initialized: now 1786241604043759 us; error 41 us; skew 500 ppm
Aug 09 02:13:24 dist-test-slave-p3t5 krb5kdc[21221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241604, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:24.046711 22357 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:13:24.047219 22357 webserver.cc:460] Webserver started at http://127.5.9.190:43637/ using document root <none> and password file <none>
I20260809 02:13:24.047513 22357 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:24.047595 22357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:24.049109 22357 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:13:24.049943 22373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:24.050134 22357 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:24.050252 22357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "c0bb51080df2466fab6e88e4f202abf9"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "8ea43e9306e6cd2038ae5d57faa49072"
server_key_iv: "c1027700b005f436ee8a822b323aa0ca"
server_key_version: "encryptionkey@0"
I20260809 02:13:24.050673 22357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:24.063618 22357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:24.066046 22357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:24.066233 22357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:24.070588 22357 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:43039
I20260809 02:13:24.070629 22429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:43039 every 8 connection(s)
I20260809 02:13:24.071043 22357 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:13:24.072652 22430 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:13:24.073360 22430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Bootstrap starting.
I20260809 02:13:24.081308 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 22357
I20260809 02:13:24.081781 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:33625
--local_ip_for_outbound_sockets=127.5.9.129
--tserver_master_addrs=127.5.9.190:43039
--webserver_port=35753
--webserver_interface=127.5.9.129
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:13:24.082804 22430 log.cc:824] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:24.085736 22430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:13:24.086081 22430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Bootstrap complete.
I20260809 02:13:24.088521 22430 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } }
I20260809 02:13:24.088754 22430 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0bb51080df2466fab6e88e4f202abf9, State: Initialized, Role: FOLLOWER
I20260809 02:13:24.088932 22430 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } }
I20260809 02:13:24.089008 22430 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:13:24.089071 22430 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:13:24.089121 22430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 02:13:24.090310 22430 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } }
I20260809 02:13:24.090402 22430 leader_election.cc:302] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0bb51080df2466fab6e88e4f202abf9; no voters:
I20260809 02:13:24.090620 22430 leader_election.cc:288] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 02:13:24.090663 22434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 02:13:24.090826 22434 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [term 3 LEADER]: Becoming Leader. State: Replica: c0bb51080df2466fab6e88e4f202abf9, State: Running, Role: LEADER
I20260809 02:13:24.090914 22430 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:13:24.090916 22434 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } }
I20260809 02:13:24.091354 22434 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0bb51080df2466fab6e88e4f202abf9. Latest consensus state: current_term: 3 leader_uuid: "c0bb51080df2466fab6e88e4f202abf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } } }
I20260809 02:13:24.091455 22434 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:24.091363 22436 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c0bb51080df2466fab6e88e4f202abf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb51080df2466fab6e88e4f202abf9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43039 } } }
I20260809 02:13:24.091665 22436 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:24.091679 22441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:13:24.092815 22441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:13:24.092954 22441 catalog_manager.cc:1259] Loaded cluster ID: d634e3913d2d4449a13e7abb4b2e879a
I20260809 02:13:24.093181 22441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:13:24.094612 22441 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:13:24.094822 22441 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Loaded TSK: 0
I20260809 02:13:24.094870 22441 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c0bb51080df2466fab6e88e4f202abf9: Loaded TSK: 1
F20260809 02:13:24.095134 22441 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fbd282a69ad google::LogMessage::Fail() at ??:0
@ 0x7fbd282a988a google::LogMessage::SendToLog() at ??:0
@ 0x7fbd282a64de google::LogMessage::Flush() at ??:0
@ 0x7fbd282a7a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fbd2aa13104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fbd2aa10d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fbd302001cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fbd3020048e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fbd30200594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fbd301c6c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fbd301d0ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fbd2a1740d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fbd2a169954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fbd2afcd6db start_thread at ??:0
@ 0x7fbd26da461f clone at ??:0
W20260809 02:13:24.187734 22433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:24.188017 22433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:24.188063 22433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:24.189349 22433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:24.189428 22433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:24.189478 22433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:13:24.191171 22433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:33625
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=35753
--tserver_master_addrs=127.5.9.190:43039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:24.191485 22433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:24.191743 22433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:24.197126 22473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:24.197373 22433 server_base.cc:1034] running on GCE node
W20260809 02:13:24.197408 22475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:24.197079 22472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:24.197714 22433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:24.198009 22433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:24.199206 22433 hybrid_clock.cc:648] HybridClock initialized: now 1786241604199177 us; error 51 us; skew 500 ppm
Aug 09 02:13:24 dist-test-slave-p3t5 krb5kdc[21221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241604, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:24.202417 22433 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:13:24.202967 22433 webserver.cc:460] Webserver started at http://127.5.9.129:35753/ using document root <none> and password file <none>
I20260809 02:13:24.203222 22433 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:24.203308 22433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:24.204838 22433 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:24.205649 22484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:24.205925 22433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:24.206044 22433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "9abc6e65e7dd48c7a4447cdccd69e49c"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "3dd8110918d3014d6b4fff3a459a882c"
server_key_iv: "64691f4b1c002450aa156cb495ac8255"
server_key_version: "encryptionkey@0"
I20260809 02:13:24.206442 22433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:24.218556 22433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:24.221082 22433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:24.221375 22433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:24.221724 22433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:24.222281 22491 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:13:24.224377 22433 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:13:24.224478 22433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 02:13:24.224530 22433 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:13:24.225255 22433 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:13:24.225332 22433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 02:13:24.225400 22491 tablet_bootstrap.cc:492] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c: Bootstrap starting.
I20260809 02:13:24.233011 22433 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:33625
I20260809 02:13:24.233157 22602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:33625 every 8 connection(s)
I20260809 02:13:24.233475 22433 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:13:24.237569 22491 log.cc:824] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:24.238977 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 22433
I20260809 02:13:24.239302 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:43693
--local_ip_for_outbound_sockets=127.5.9.130
--tserver_master_addrs=127.5.9.190:43039
--webserver_port=35547
--webserver_interface=127.5.9.130
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:13:24.240553 22491 tablet_bootstrap.cc:492] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:13:24.240923 22491 tablet_bootstrap.cc:492] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c: Bootstrap complete.
I20260809 02:13:24.242290 22491 ts_tablet_manager.cc:1397] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.007s
I20260809 02:13:24.244096 22491 raft_consensus.cc:348] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } }
I20260809 02:13:24.244486 22491 raft_consensus.cc:729] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9abc6e65e7dd48c7a4447cdccd69e49c, State: Initialized, Role: FOLLOWER
I20260809 02:13:24.244664 22491 consensus_queue.cc:260] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } }
I20260809 02:13:24.245009 22491 ts_tablet_manager.cc:1428] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
W20260809 02:13:24.315958 22605 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:13:24.234264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:37837)
0809 02:13:24.234465 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:24.234507 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:24.235051 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:24.315710 (+ 80659us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":106,"threads_started":1}
W20260809 02:13:24.316213 22603 heartbeater.cc:644] Failed to heartbeat to 127.5.9.190:43039 (0 consecutive failures): Network error: Failed to ping master at 127.5.9.190:43039: Client connection negotiation failed: client connection to 127.5.9.190:43039: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 02:13:24.316598 22605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:24.316463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:59791)
0809 02:13:24.316523 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:24.316569 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 02:13:24.316991 22605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:24.316890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:59323)
0809 02:13:24.316947 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:24.316974 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 02:13:24.317131 22603 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:13:24.330375 22607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:24.330648 22607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:24.330693 22607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:24.332024 22607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:24.332113 22607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:24.332162 22607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:13:24.333905 22607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:43693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=35547
--tserver_master_addrs=127.5.9.190:43039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:24.334213 22607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:24.334472 22607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:13:24.340241 22607 server_base.cc:1034] running on GCE node
W20260809 02:13:24.340194 22615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:24.340186 22614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:24.340175 22617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:24.340698 22607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:24.341012 22607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:24.342185 22607 hybrid_clock.cc:648] HybridClock initialized: now 1786241604342150 us; error 67 us; skew 500 ppm
Aug 09 02:13:24 dist-test-slave-p3t5 krb5kdc[21221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241604, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:24.345014 22607 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:13:24.345571 22607 webserver.cc:460] Webserver started at http://127.5.9.130:35547/ using document root <none> and password file <none>
I20260809 02:13:24.345858 22607 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:24.345944 22607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:24.347432 22607 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:24.348119 22624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:24.348336 22607 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:24.348446 22607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "83b551ca410f45fdb9c82efbae122086"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "d95b63bec94b61eb062527c8e6b524ff"
server_key_iv: "edcf6a88c568493fa891b3557652a353"
server_key_version: "encryptionkey@0"
I20260809 02:13:24.348831 22607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:24.363613 22607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:24.365969 22607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:24.366155 22607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:24.366415 22607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:24.366768 22607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:24.366828 22607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:24.366870 22607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:24.366900 22607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:24.373242 22607 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:43693
I20260809 02:13:24.373294 22747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:43693 every 8 connection(s)
I20260809 02:13:24.373721 22607 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:13:24.374820 22750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:24.374424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:48509)
0809 02:13:24.374596 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:24.374672 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
W20260809 02:13:24.375226 22748 heartbeater.cc:644] Failed to heartbeat to 127.5.9.190:43039 (0 consecutive failures): Network error: Failed to ping master at 127.5.9.190:43039: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
I20260809 02:13:24.375500 22750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:24.375404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:43187)
0809 02:13:24.375454 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:24.375482 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 02:13:24.375744 22750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:24.375694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:50499)
0809 02:13:24.375714 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:24.375732 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260809 02:13:24.375842 22748 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 02:13:24.376147 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 22607
I20260809 02:13:24.376569 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:41333
--local_ip_for_outbound_sockets=127.5.9.131
--tserver_master_addrs=127.5.9.190:43039
--webserver_port=42981
--webserver_interface=127.5.9.131
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:24.463963 22751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:24.464210 22751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:24.464237 22751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:24.465423 22751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:24.465466 22751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:24.465492 22751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:13:24.467144 22751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:41333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=42981
--tserver_master_addrs=127.5.9.190:43039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:24.467372 22751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:24.467583 22751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:24.473019 22757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:24.473011 22759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:24.473030 22751 server_base.cc:1034] running on GCE node
W20260809 02:13:24.473011 22756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:24.473642 22751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:24.473887 22751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:24.475069 22751 hybrid_clock.cc:648] HybridClock initialized: now 1786241604475049 us; error 39 us; skew 500 ppm
Aug 09 02:13:24 dist-test-slave-p3t5 krb5kdc[21221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241604, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:24.477751 22751 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:13:24.478215 22751 webserver.cc:460] Webserver started at http://127.5.9.131:42981/ using document root <none> and password file <none>
I20260809 02:13:24.478487 22751 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:24.478561 22751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:24.479954 22751 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:13:24.482494 22767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:24.482692 22751 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:24.482796 22751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "398b3a4c775d4bdfa1cd79af9e4b603c"
format_stamp: "Formatted at 2026-08-09 02:12:42 on dist-test-slave-p3t5"
server_key: "c74d7d423f4908d71ac7f716737162d6"
server_key_iv: "42af483b6652352c8115af7d64e04f9d"
server_key_version: "encryptionkey@0"
I20260809 02:13:24.483191 22751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:24.500005 22751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:24.503404 22751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:24.503590 22751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:24.503861 22751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:24.504333 22751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:24.504381 22751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:24.504441 22751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:24.504462 22751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:24.511488 22751 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:41333
I20260809 02:13:24.511569 22885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:41333 every 8 connection(s)
I20260809 02:13:24.511940 22751 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:13:24.512974 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 22751
I20260809 02:13:24.513157 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:24.512687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:40999)
0809 02:13:24.512883 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:24.512964 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":112,"threads_started":1}
W20260809 02:13:24.513700 22886 heartbeater.cc:644] Failed to heartbeat to 127.5.9.190:43039 (0 consecutive failures): Network error: Failed to ping master at 127.5.9.190:43039: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
I20260809 02:13:24.514019 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:24.513915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:34399)
0809 02:13:24.513968 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:24.513997 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 02:13:24.514415 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:24.514332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:45967)
0809 02:13:24.514376 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:24.514399 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 02:13:24.514587 22886 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 02:13:25.317991 22890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:25.317601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:55103)
0809 02:13:25.317864 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:25.317913 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":129,"threads_started":1}
I20260809 02:13:25.376499 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:25.376267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:52349)
0809 02:13:25.376409 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:25.376445 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
I20260809 02:13:25.515189 22892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:25.514964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:43077)
0809 02:13:25.515090 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:25.515124 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260809 02:13:25.952107 22893 raft_consensus.cc:482] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:13:25.952270 22893 raft_consensus.cc:504] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } }
I20260809 02:13:25.952636 22893 leader_election.cc:288] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83b551ca410f45fdb9c82efbae122086 (127.5.9.130:43693), 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333)
Aug 09 02:13:25 dist-test-slave-p3t5 krb5kdc[21221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241604, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.130@KRBTEST.COM
Aug 09 02:13:25 dist-test-slave-p3t5 krb5kdc[21221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241604, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.131@KRBTEST.COM
I20260809 02:13:25.968149 22895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:25.952844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:41333 (local address 127.5.9.129:37069)
0809 02:13:25.953120 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:25.953129 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:25.953176 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:25.954071 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:25.954074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:25.954665 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:25.954966 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:25.954974 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:25.956697 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:25.956698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:25.958114 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:25.958118 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:25.959028 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:25.959030 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:25.959163 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:25.960997 (+ 1834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:13:25.961005 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:25.963299 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:25.966063 (+ 2764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:25.966073 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:25.966076 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:25.966457 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:25.966807 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:25.966808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:25.966808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:25.966958 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:25.967521 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:25.967529 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:25.967694 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 02:13:25.967809 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 02:13:25.968008 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":86,"thread_start_us":50,"threads_started":1}
I20260809 02:13:25.969043 22894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:25.952808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:43693 (local address 127.5.9.129:53809)
0809 02:13:25.952967 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:25.952979 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:25.953046 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:25.954071 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:25.954074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:25.954671 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:25.954966 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:25.954974 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:25.956698 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:25.956699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:25.958114 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:25.958118 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:25.959131 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:25.959133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:25.959219 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:25.960005 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:13:25.960016 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:25.963122 (+ 3106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:25.967565 (+ 4443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:25.967566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:25.967566 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:25.967808 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:25.968126 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:25.968127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:25.968127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:25.968188 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:25.968695 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:25.968696 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:25.968747 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 02:13:25.968824 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 02:13:25.968938 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":92,"thread_start_us":75,"threads_started":1}
I20260809 02:13:25.969077 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:25.952979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:37069 (local address 127.5.9.131:41333)
0809 02:13:25.953144 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:25.953153 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:25.953193 (+ 40us) server_negotiation.cc:407] Connection header received
0809 02:13:25.953768 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:25.953772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:25.953833 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:25.954001 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:25.955126 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:25.956570 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:25.958218 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:25.958889 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:25.959004 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:25.963428 (+ 4424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:25.963434 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:25.963435 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:25.963451 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:25.965956 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:25.966572 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:25.966576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:25.966576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:25.966704 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:25.967040 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:25.967041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:25.967042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:25.967408 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:25.967489 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:25.968812 (+ 1323us) server_negotiation.cc:299] Negotiation successful
0809 02:13:25.968965 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":60,"threads_started":1}
I20260809 02:13:25.969355 22896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:25.952942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:53809 (local address 127.5.9.130:43693)
0809 02:13:25.953092 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:25.953105 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:25.953133 (+ 28us) server_negotiation.cc:407] Connection header received
0809 02:13:25.953768 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:25.953772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:25.953834 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:25.954001 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:25.955126 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:25.956570 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:25.958218 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:25.958888 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:25.959004 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:25.964838 (+ 5834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:25.964843 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:25.964844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:25.964895 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:25.967311 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:25.967932 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:25.967935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:25.967936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:25.968049 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:25.968260 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:25.968261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:25.968261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:25.968579 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:25.968662 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:25.968940 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 02:13:25.969106 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
W20260809 02:13:25.970224 22487 leader_election.cc:341] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83b551ca410f45fdb9c82efbae122086 (127.5.9.130:43693): Not found: Tablet not found: b0ee32a24b1741bb8cc408062f3edee3
W20260809 02:13:25.970472 22488 leader_election.cc:341] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333): Not found: Tablet not found: b0ee32a24b1741bb8cc408062f3edee3
I20260809 02:13:25.970558 22488 leader_election.cc:302] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9abc6e65e7dd48c7a4447cdccd69e49c; no voters: 398b3a4c775d4bdfa1cd79af9e4b603c, 83b551ca410f45fdb9c82efbae122086
I20260809 02:13:25.970700 22893 raft_consensus.cc:2726] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:13:26.318832 22894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:26.318603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:56123)
0809 02:13:26.318710 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:26.318772 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 02:13:26.377341 22898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:26.377061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:39395)
0809 02:13:26.377221 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:26.377284 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
I20260809 02:13:26.515944 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:26.515694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:47677)
0809 02:13:26.515837 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:26.515890 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260809 02:13:27.319635 22900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:27.319373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:46219)
0809 02:13:27.319535 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:27.319583 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":79,"threads_started":1}
I20260809 02:13:27.378250 22901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:27.377995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:33679)
0809 02:13:27.378155 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:27.378201 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":73,"threads_started":1}
I20260809 02:13:27.503593 22902 raft_consensus.cc:482] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:13:27.503710 22902 raft_consensus.cc:504] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } }
I20260809 02:13:27.503964 22902 leader_election.cc:288] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83b551ca410f45fdb9c82efbae122086 (127.5.9.130:43693), 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333)
W20260809 02:13:27.504539 22488 leader_election.cc:341] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333): Not found: Tablet not found: b0ee32a24b1741bb8cc408062f3edee3
W20260809 02:13:27.504635 22487 leader_election.cc:341] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83b551ca410f45fdb9c82efbae122086 (127.5.9.130:43693): Not found: Tablet not found: b0ee32a24b1741bb8cc408062f3edee3
I20260809 02:13:27.504716 22487 leader_election.cc:302] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9abc6e65e7dd48c7a4447cdccd69e49c; no voters: 398b3a4c775d4bdfa1cd79af9e4b603c, 83b551ca410f45fdb9c82efbae122086
I20260809 02:13:27.504841 22902 raft_consensus.cc:2726] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:13:27.516757 22903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:27.516496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:33237)
0809 02:13:27.516658 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:27.516707 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260809 02:13:28.320547 22904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:28.320283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:58291)
0809 02:13:28.320442 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:28.320486 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":75,"threads_started":1}
I20260809 02:13:28.379124 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:28.378848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:49457)
0809 02:13:28.379017 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:28.379070 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":69,"threads_started":1}
I20260809 02:13:28.517598 22906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:28.517271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:33639)
0809 02:13:28.517448 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:28.517533 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260809 02:13:29.321421 22907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:29.321176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:40251)
0809 02:13:29.321320 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:29.321367 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
I20260809 02:13:29.379956 22908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:29.379723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:57853)
0809 02:13:29.379869 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:29.379907 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
I20260809 02:13:29.518340 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:29.518090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:51219)
0809 02:13:29.518231 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:29.518277 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260809 02:13:29.856451 22910 raft_consensus.cc:482] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:13:29.856595 22910 raft_consensus.cc:504] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } }
I20260809 02:13:29.856827 22910 leader_election.cc:288] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83b551ca410f45fdb9c82efbae122086 (127.5.9.130:43693), 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333)
W20260809 02:13:29.857286 22487 leader_election.cc:341] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83b551ca410f45fdb9c82efbae122086 (127.5.9.130:43693): Not found: Tablet not found: b0ee32a24b1741bb8cc408062f3edee3
W20260809 02:13:29.857383 22488 leader_election.cc:341] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333): Not found: Tablet not found: b0ee32a24b1741bb8cc408062f3edee3
I20260809 02:13:29.857466 22488 leader_election.cc:302] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9abc6e65e7dd48c7a4447cdccd69e49c; no voters: 398b3a4c775d4bdfa1cd79af9e4b603c, 83b551ca410f45fdb9c82efbae122086
I20260809 02:13:29.857551 22910 raft_consensus.cc:2726] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:13:30.322359 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:30.322061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:46913)
0809 02:13:30.322231 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:30.322302 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
I20260809 02:13:30.380723 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:30.380459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:42267)
0809 02:13:30.380631 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:30.380674 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":84,"threads_started":1}
I20260809 02:13:30.519147 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:30.518896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:51967)
0809 02:13:30.519053 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:30.519095 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260809 02:13:31.323247 22914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:31.322999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:51731)
0809 02:13:31.323159 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:31.323194 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
I20260809 02:13:31.381489 22915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:31.381256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:39617)
0809 02:13:31.381392 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:31.381437 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260809 02:13:31.520045 22916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:31.519765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:39549)
0809 02:13:31.519922 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:31.519972 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":69,"threads_started":1}
I20260809 02:13:32.324115 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:32.323848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:50433)
0809 02:13:32.324014 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:32.324060 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":82,"threads_started":1}
I20260809 02:13:32.382272 22918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:32.382024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:55071)
0809 02:13:32.382180 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:32.382222 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260809 02:13:32.520826 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:32.520596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:55369)
0809 02:13:32.520738 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:32.520774 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260809 02:13:33.325109 22920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:33.324858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:51203)
0809 02:13:33.325001 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:33.325051 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":71,"threads_started":1}
I20260809 02:13:33.383165 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:33.382939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:53789)
0809 02:13:33.383073 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:33.383116 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260809 02:13:33.521590 22922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:33.521341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:40915)
0809 02:13:33.521491 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:33.521536 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260809 02:13:34.326002 22923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:34.325741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:54297)
0809 02:13:34.325910 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:34.325949 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":105,"threads_started":1}
I20260809 02:13:34.384080 22924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:34.383813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:60523)
0809 02:13:34.383962 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:34.384008 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260809 02:13:34.522509 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:34.522195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:39665)
0809 02:13:34.522357 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:34.522423 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":78,"threads_started":1}
I20260809 02:13:34.845991 22927 raft_consensus.cc:482] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:13:34.846096 22927 raft_consensus.cc:504] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } }
I20260809 02:13:34.846319 22927 leader_election.cc:288] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83b551ca410f45fdb9c82efbae122086 (127.5.9.130:43693), 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333)
W20260809 02:13:34.846765 22487 leader_election.cc:341] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83b551ca410f45fdb9c82efbae122086 (127.5.9.130:43693): Not found: Tablet not found: b0ee32a24b1741bb8cc408062f3edee3
W20260809 02:13:34.846873 22488 leader_election.cc:341] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333): Not found: Tablet not found: b0ee32a24b1741bb8cc408062f3edee3
I20260809 02:13:34.846964 22488 leader_election.cc:302] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9abc6e65e7dd48c7a4447cdccd69e49c; no voters: 398b3a4c775d4bdfa1cd79af9e4b603c, 83b551ca410f45fdb9c82efbae122086
I20260809 02:13:34.847029 22927 raft_consensus.cc:2726] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:13:35.326815 22928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:35.326545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:35327)
0809 02:13:35.326698 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:35.326758 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260809 02:13:35.385011 22929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:35.384718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:34569)
0809 02:13:35.384893 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:35.384957 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":66,"threads_started":1}
I20260809 02:13:35.523285 22930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:35.523044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:57349)
0809 02:13:35.523184 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:35.523233 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260809 02:13:36.327710 22931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:36.327414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:56741)
0809 02:13:36.327602 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:36.327655 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":80,"threads_started":1}
I20260809 02:13:36.385838 22932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:36.385577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:44319)
0809 02:13:36.385746 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:36.385786 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":95,"threads_started":1}
I20260809 02:13:36.524170 22933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:36.523892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:33117)
0809 02:13:36.524043 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:36.524093 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":68,"threads_started":1}
I20260809 02:13:36.975299 22934 raft_consensus.cc:482] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:13:36.975423 22934 raft_consensus.cc:504] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b551ca410f45fdb9c82efbae122086" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43693 } } peers { permanent_uuid: "398b3a4c775d4bdfa1cd79af9e4b603c" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 41333 } } peers { permanent_uuid: "9abc6e65e7dd48c7a4447cdccd69e49c" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 33625 } }
I20260809 02:13:36.975628 22934 leader_election.cc:288] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83b551ca410f45fdb9c82efbae122086 (127.5.9.130:43693), 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333)
W20260809 02:13:36.976082 22488 leader_election.cc:341] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 398b3a4c775d4bdfa1cd79af9e4b603c (127.5.9.131:41333): Not found: Tablet not found: b0ee32a24b1741bb8cc408062f3edee3
W20260809 02:13:36.976193 22487 leader_election.cc:341] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83b551ca410f45fdb9c82efbae122086 (127.5.9.130:43693): Not found: Tablet not found: b0ee32a24b1741bb8cc408062f3edee3
I20260809 02:13:36.976272 22487 leader_election.cc:302] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9abc6e65e7dd48c7a4447cdccd69e49c; no voters: 398b3a4c775d4bdfa1cd79af9e4b603c, 83b551ca410f45fdb9c82efbae122086
I20260809 02:13:36.976363 22934 raft_consensus.cc:2726] T b0ee32a24b1741bb8cc408062f3edee3 P 9abc6e65e7dd48c7a4447cdccd69e49c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:13:37.328634 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:37.328378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:43395)
0809 02:13:37.328530 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:37.328581 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260809 02:13:37.386823 22936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:37.386469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:55479)
0809 02:13:37.386650 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:37.386754 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":111,"threads_started":1}
I20260809 02:13:37.525161 22937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:37.524884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:38123)
0809 02:13:37.525057 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:37.525108 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":76,"threads_started":1}
I20260809 02:13:38.329445 22938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:38.329191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:34103)
0809 02:13:38.329328 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:38.329375 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260809 02:13:38.387625 22939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:38.387370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:53445)
0809 02:13:38.387498 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:38.387559 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260809 02:13:38.526047 22940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:38.525795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:39987)
0809 02:13:38.525952 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:38.525994 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260809 02:13:39.330402 22941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:39.330122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.129:44033)
0809 02:13:39.330299 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:39.330348 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":77,"threads_started":1}
I20260809 02:13:39.388412 22942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:39.388168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.130:52837)
0809 02:13:39.388318 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:39.388363 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":64,"threads_started":1}
I20260809 02:13:39.522836 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 22433
I20260809 02:13:39.526911 22943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:39.526657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:43039 (local address 127.5.9.131:41813)
0809 02:13:39.526806 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:39.526853 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.190:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":101,"threads_started":1}
I20260809 02:13:39.530570 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 22607
I20260809 02:13:39.538734 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 22751
2026-08-09T02:13:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58654 ms)
[----------] 30 tests from SecurityITest (242146 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:13:39 dist-test-slave-p3t5 krb5kdc[22947](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:13:39 dist-test-slave-p3t5 krb5kdc[22947](info): set up 2 sockets
Aug 09 02:13:39 dist-test-slave-p3t5 krb5kdc[22947](info): commencing operation
krb5kdc: starting...
W20260809 02:13:40.629915 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:13:40 dist-test-slave-p3t5 krb5kdc[22947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:13:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:13:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:13:40.762902 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:41671
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:41671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:40.848312 22961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:40.848593 22961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:40.848620 22961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:40.849962 22961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:13:40.850015 22961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:40.850030 22961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:40.850044 22961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:13:40.850055 22961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:13:40.851734 22961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:41671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:41671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:40.852042 22961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:40.852313 22961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:40.858146 22969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:40.858131 22967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:40.858174 22961 server_base.cc:1034] running on GCE node
W20260809 02:13:40.858139 22966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:40.858625 22961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:40.858981 22961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:40.860152 22961 hybrid_clock.cc:648] HybridClock initialized: now 1786241620860127 us; error 39 us; skew 500 ppm
Aug 09 02:13:40 dist-test-slave-p3t5 krb5kdc[22947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241620, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:40.863184 22961 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:13:40.863715 22961 webserver.cc:460] Webserver started at http://127.5.9.190:40655/ using document root <none> and password file <none>
I20260809 02:13:40.864003 22961 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:40.864053 22961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:40.864250 22961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:40.865396 22961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "9b602adb65b143b49093ed400e25d9a9"
format_stamp: "Formatted at 2026-08-09 02:13:40 on dist-test-slave-p3t5"
server_key: "6316706cd11bda0554cbc1c6505eecdc"
server_key_iv: "2d117260e294d743437423d340fad1d3"
server_key_version: "encryptionkey@0"
I20260809 02:13:40.865806 22961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "9b602adb65b143b49093ed400e25d9a9"
format_stamp: "Formatted at 2026-08-09 02:13:40 on dist-test-slave-p3t5"
server_key: "6316706cd11bda0554cbc1c6505eecdc"
server_key_iv: "2d117260e294d743437423d340fad1d3"
server_key_version: "encryptionkey@0"
I20260809 02:13:40.867286 22961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:13:40.868076 22977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:40.868260 22961 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:13:40.868379 22961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "9b602adb65b143b49093ed400e25d9a9"
format_stamp: "Formatted at 2026-08-09 02:13:40 on dist-test-slave-p3t5"
server_key: "6316706cd11bda0554cbc1c6505eecdc"
server_key_iv: "2d117260e294d743437423d340fad1d3"
server_key_version: "encryptionkey@0"
I20260809 02:13:40.868460 22961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:40.882730 22961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:40.885340 22961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:40.885543 22961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:40.889956 22961 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:41671
I20260809 02:13:40.890015 23031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:41671 every 8 connection(s)
I20260809 02:13:40.890424 22961 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:13:40.891338 23032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:40.893780 23032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9: Bootstrap starting.
I20260809 02:13:40.894423 23032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:40.894749 23032 log.cc:824] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:40.895545 23032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9: No bootstrap required, opened a new log
I20260809 02:13:40.897197 23032 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b602adb65b143b49093ed400e25d9a9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41671 } }
I20260809 02:13:40.897349 23032 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:40.897372 23032 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b602adb65b143b49093ed400e25d9a9, State: Initialized, Role: FOLLOWER
I20260809 02:13:40.897569 23032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b602adb65b143b49093ed400e25d9a9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41671 } }
I20260809 02:13:40.897665 23032 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:13:40.897725 23032 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:13:40.897773 23032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:40.898402 23032 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b602adb65b143b49093ed400e25d9a9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41671 } }
I20260809 02:13:40.898528 23032 leader_election.cc:302] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b602adb65b143b49093ed400e25d9a9; no voters:
I20260809 02:13:40.898712 23032 leader_election.cc:288] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:13:40.898763 23035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:40.898975 23035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 9b602adb65b143b49093ed400e25d9a9, State: Running, Role: LEADER
I20260809 02:13:40.899019 23032 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:13:40.899168 23035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b602adb65b143b49093ed400e25d9a9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41671 } }
I20260809 02:13:40.899228 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 22961
I20260809 02:13:40.899331 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:13:40.899564 5158 external_mini_cluster.cc:1428] Setting key 493c5a46fb31f02f7ee1ebec7a74c6f6
I20260809 02:13:40.899861 23036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b602adb65b143b49093ed400e25d9a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b602adb65b143b49093ed400e25d9a9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41671 } } }
I20260809 02:13:40.899978 23036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:40.900005 23038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b602adb65b143b49093ed400e25d9a9. Latest consensus state: current_term: 1 leader_uuid: "9b602adb65b143b49093ed400e25d9a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b602adb65b143b49093ed400e25d9a9" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41671 } } }
I20260809 02:13:40.900085 23038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:40.900497 23045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:13:40.901446 23045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:13:40.903033 23045 catalog_manager.cc:1347] Generated new cluster ID: eeaa5b53c2d64706aec7d06db40c6ce2
I20260809 02:13:40.903090 23045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:13:40 dist-test-slave-p3t5 krb5kdc[22947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:13:40.907836 23045 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:13:40.908551 23045 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:13:40.912577 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:40.900289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57832 (local address 127.5.9.190:41671)
0809 02:13:40.900458 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:40.900468 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:40.900519 (+ 51us) server_negotiation.cc:407] Connection header received
0809 02:13:40.900647 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:40.900655 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:40.900716 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:40.901733 (+ 1017us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:40.901848 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:40.903182 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:40.904259 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:40.904873 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:40.904986 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:40.908174 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:40.908178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:40.908179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:40.908199 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:40.910835 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:40.911272 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:40.911275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:40.911276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:40.911397 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:40.911644 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:40.911645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:40.911645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:40.911889 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:40.912067 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:40.912174 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 02:13:40.912355 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
I20260809 02:13:40.915851 23045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9: Generated new TSK 0
I20260809 02:13:40.916065 23045 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:13:40.954885 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41671
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:41.063351 23059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:41.063588 23059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:41.063616 23059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:41.064918 23059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:41.064977 23059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:41.065006 23059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:13:41.066613 23059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:41.066905 23059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:41.067166 23059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:41.073207 23065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:41.073226 23066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:41.073207 23068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:41.073418 23059 server_base.cc:1034] running on GCE node
I20260809 02:13:41.073765 23059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:41.074035 23059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:41.075212 23059 hybrid_clock.cc:648] HybridClock initialized: now 1786241621075189 us; error 43 us; skew 500 ppm
Aug 09 02:13:41 dist-test-slave-p3t5 krb5kdc[22947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241621, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:41.077922 23059 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:13:41.078339 23059 webserver.cc:460] Webserver started at http://127.5.9.129:44531/ using document root <none> and password file <none>
I20260809 02:13:41.078539 23059 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:41.078583 23059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:41.078682 23059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:41.079725 23059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "3b1e56f0ef8f4a4eba243ee6802f343b"
format_stamp: "Formatted at 2026-08-09 02:13:41 on dist-test-slave-p3t5"
server_key: "9843804775880ebf31670b365455a74f"
server_key_iv: "8a9c68e21de3fec8b7668dc171792eb3"
server_key_version: "encryptionkey@0"
I20260809 02:13:41.080049 23059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "3b1e56f0ef8f4a4eba243ee6802f343b"
format_stamp: "Formatted at 2026-08-09 02:13:41 on dist-test-slave-p3t5"
server_key: "9843804775880ebf31670b365455a74f"
server_key_iv: "8a9c68e21de3fec8b7668dc171792eb3"
server_key_version: "encryptionkey@0"
I20260809 02:13:41.081374 23059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:13:41.082191 23075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:41.082381 23059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:13:41.082469 23059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "3b1e56f0ef8f4a4eba243ee6802f343b"
format_stamp: "Formatted at 2026-08-09 02:13:41 on dist-test-slave-p3t5"
server_key: "9843804775880ebf31670b365455a74f"
server_key_iv: "8a9c68e21de3fec8b7668dc171792eb3"
server_key_version: "encryptionkey@0"
I20260809 02:13:41.082530 23059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:41.108687 23059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:41.110803 23059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:41.110960 23059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:41.111191 23059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:41.111557 23059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:41.111593 23059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:41.111644 23059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:41.111670 23059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:41.117663 23059 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:46533
I20260809 02:13:41.117767 23195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:46533 every 8 connection(s)
I20260809 02:13:41.118108 23059 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:13:41.122713 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 23059
I20260809 02:13:41.122838 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:13:41.123065 5158 external_mini_cluster.cc:1428] Setting key b269aa6d5fa224951b4d211c7e7f8d65
Aug 09 02:13:41 dist-test-slave-p3t5 krb5kdc[22947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241621, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:13:41.129166 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.118929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:56239 (local address 127.5.9.190:41671)
0809 02:13:41.119016 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:41.119018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:41.119185 (+ 167us) server_negotiation.cc:407] Connection header received
0809 02:13:41.119671 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:41.119672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:41.119686 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:41.119748 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:41.120732 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.121249 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.122518 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.122840 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.122914 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.126082 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:41.126084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:41.126084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:41.126092 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:41.127658 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:41.128119 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:41.128121 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:41.128121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:41.128188 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:41.128458 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:41.128459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:41.128459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:41.128582 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:41.128657 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:41.128959 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 02:13:41.129068 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:13:41.129359 23198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.118876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:41671 (local address 127.5.9.129:56239)
0809 02:13:41.119042 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:41.119087 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:41.119581 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:41.119803 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:41.119806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:41.120415 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:41.120627 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:41.120632 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.121384 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.121385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:41.122394 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:41.122397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.122968 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.122969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:41.123078 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.123872 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:13:41.123884 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:41.125950 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:41.127767 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:41.127775 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:41.127777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:41.128024 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:41.128265 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:41.128266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:41.128267 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:41.128385 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:41.128660 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:41.128667 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:41.128817 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 02:13:41.128922 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 02:13:41.129102 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260809 02:13:41.129940 23196 heartbeater.cc:347] Connected to a master server at 127.5.9.190:41671
I20260809 02:13:41.130029 23196 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:41.130237 23196 heartbeater.cc:511] Master 127.5.9.190:41671 requested a full tablet report, sending...
I20260809 02:13:41.130854 22994 ts_manager.cc:194] Registered new tserver with Master: 3b1e56f0ef8f4a4eba243ee6802f343b (127.5.9.129:46533)
I20260809 02:13:41.131858 22994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:56239
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:13:41.152626 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41671
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:41.238421 23201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:41.238646 23201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:41.238672 23201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:41.239773 23201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:41.239809 23201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:41.239835 23201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:13:41.241223 23201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:41.241430 23201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:41.241611 23201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:41.246928 23207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:41.247105 23201 server_base.cc:1034] running on GCE node
W20260809 02:13:41.246932 23209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:41.247005 23206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:41.247462 23201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:41.247675 23201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:41.248811 23201 hybrid_clock.cc:648] HybridClock initialized: now 1786241621248789 us; error 38 us; skew 500 ppm
Aug 09 02:13:41 dist-test-slave-p3t5 krb5kdc[22947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241621, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:41.251451 23201 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:13:41.251853 23201 webserver.cc:460] Webserver started at http://127.5.9.130:36843/ using document root <none> and password file <none>
I20260809 02:13:41.252112 23201 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:41.252184 23201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:41.252334 23201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:41.253409 23201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "a22a654a9bb049c4a2bfe07581e58812"
format_stamp: "Formatted at 2026-08-09 02:13:41 on dist-test-slave-p3t5"
server_key: "be7686f6fb6b43321667382cb3de0b43"
server_key_iv: "fcfdd295730216b7f2e5003adc6287c8"
server_key_version: "encryptionkey@0"
I20260809 02:13:41.253808 23201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "a22a654a9bb049c4a2bfe07581e58812"
format_stamp: "Formatted at 2026-08-09 02:13:41 on dist-test-slave-p3t5"
server_key: "be7686f6fb6b43321667382cb3de0b43"
server_key_iv: "fcfdd295730216b7f2e5003adc6287c8"
server_key_version: "encryptionkey@0"
I20260809 02:13:41.255177 23201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:13:41.255939 23217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:41.256119 23201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 02:13:41.256230 23201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "a22a654a9bb049c4a2bfe07581e58812"
format_stamp: "Formatted at 2026-08-09 02:13:41 on dist-test-slave-p3t5"
server_key: "be7686f6fb6b43321667382cb3de0b43"
server_key_iv: "fcfdd295730216b7f2e5003adc6287c8"
server_key_version: "encryptionkey@0"
I20260809 02:13:41.256309 23201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:41.269847 23201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:41.272317 23201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:41.272490 23201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:41.272744 23201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:41.273080 23201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:41.273137 23201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:41.273185 23201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:41.273207 23201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:41.279414 23201 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:37111
I20260809 02:13:41.279496 23337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:37111 every 8 connection(s)
I20260809 02:13:41.279834 23201 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:13:41 dist-test-slave-p3t5 krb5kdc[22947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241621, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:13:41.288820 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 23201
I20260809 02:13:41.288959 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:13:41.289223 5158 external_mini_cluster.cc:1428] Setting key 945cacdcd14169183c4d120699f42169
I20260809 02:13:41.289728 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.280619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:33521 (local address 127.5.9.190:41671)
0809 02:13:41.280664 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:41.280666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:41.280899 (+ 233us) server_negotiation.cc:407] Connection header received
0809 02:13:41.281385 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:41.281386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:41.281397 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:41.281449 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:41.282430 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.282955 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.284049 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.284253 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.284321 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.286774 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:41.286776 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:41.286776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:41.286783 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:41.288089 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:41.288483 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:41.288484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:41.288484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:41.288540 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:41.288875 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:41.288876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:41.288876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:41.289082 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:41.289137 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:41.289503 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 02:13:41.289605 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:13:41.289958 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.280607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:41671 (local address 127.5.9.130:33521)
0809 02:13:41.280771 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:41.280809 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:41.281297 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:41.281511 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:41.281515 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:41.282139 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:41.282344 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:41.282350 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.283073 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.283074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:41.283945 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:41.283948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.284354 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.284355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:41.284477 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.285078 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:13:41.285087 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:41.286678 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:41.288169 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:41.288175 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:41.288178 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:41.288411 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:41.288626 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:41.288627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:41.288628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:41.288804 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:41.289136 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:41.289144 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:41.289331 (+ 187us) client_negotiation.cc:769] Sending connection context
0809 02:13:41.289458 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 02:13:41.289687 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260809 02:13:41.290416 23338 heartbeater.cc:347] Connected to a master server at 127.5.9.190:41671
I20260809 02:13:41.290532 23338 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:41.290738 23338 heartbeater.cc:511] Master 127.5.9.190:41671 requested a full tablet report, sending...
I20260809 02:13:41.291080 22994 ts_manager.cc:194] Registered new tserver with Master: a22a654a9bb049c4a2bfe07581e58812 (127.5.9.130:37111)
I20260809 02:13:41.291464 22994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:33521
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:13:41.319085 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41671
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:41.414354 23343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:41.414615 23343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:41.414644 23343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:41.416143 23343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:41.416196 23343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:41.416229 23343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:13:41.418087 23343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:41.418411 23343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:41.418681 23343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:13:41.424383 23343 server_base.cc:1034] running on GCE node
W20260809 02:13:41.424396 23348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:41.424425 23351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:41.424396 23349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:41.424934 23343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:41.425268 23343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:41.426434 23343 hybrid_clock.cc:648] HybridClock initialized: now 1786241621426427 us; error 39 us; skew 500 ppm
Aug 09 02:13:41 dist-test-slave-p3t5 krb5kdc[22947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241621, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:41.429184 23343 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:13:41.429714 23343 webserver.cc:460] Webserver started at http://127.5.9.131:42301/ using document root <none> and password file <none>
I20260809 02:13:41.429960 23343 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:41.430011 23343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:41.430119 23343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:41.431305 23343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "d2004d6845404f01a842cc9ded404f39"
format_stamp: "Formatted at 2026-08-09 02:13:41 on dist-test-slave-p3t5"
server_key: "c1ee50b7a87dacbf61899e973e3a3552"
server_key_iv: "8f0ed89dc4a72a62aed945581e9bfcc2"
server_key_version: "encryptionkey@0"
I20260809 02:13:41.431654 23343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "d2004d6845404f01a842cc9ded404f39"
format_stamp: "Formatted at 2026-08-09 02:13:41 on dist-test-slave-p3t5"
server_key: "c1ee50b7a87dacbf61899e973e3a3552"
server_key_iv: "8f0ed89dc4a72a62aed945581e9bfcc2"
server_key_version: "encryptionkey@0"
I20260809 02:13:41.433159 23343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:13:41.434092 23360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:41.434365 23343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:41.434475 23343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "d2004d6845404f01a842cc9ded404f39"
format_stamp: "Formatted at 2026-08-09 02:13:41 on dist-test-slave-p3t5"
server_key: "c1ee50b7a87dacbf61899e973e3a3552"
server_key_iv: "8f0ed89dc4a72a62aed945581e9bfcc2"
server_key_version: "encryptionkey@0"
I20260809 02:13:41.434546 23343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:41.450652 23343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:41.452809 23343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:41.452947 23343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:41.453166 23343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:41.453603 23343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:41.453722 23343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:41.453786 23343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:41.453819 23343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:41.460700 23343 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:37619
I20260809 02:13:41.460801 23476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:37619 every 8 connection(s)
I20260809 02:13:41.461171 23343 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:13:41.465888 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 23343
I20260809 02:13:41.466004 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:13:41.466217 5158 external_mini_cluster.cc:1428] Setting key ebc47a9d825786954ba3b4bd14101f78
Aug 09 02:13:41 dist-test-slave-p3t5 krb5kdc[22947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241621, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:13:41.471078 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.462059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:41033 (local address 127.5.9.190:41671)
0809 02:13:41.462130 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:41.462133 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:41.462342 (+ 209us) server_negotiation.cc:407] Connection header received
0809 02:13:41.462880 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:41.462881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:41.462893 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:41.462958 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:41.463946 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.464461 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.465513 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.465829 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.465947 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.468345 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:41.468346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:41.468346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:41.468354 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:41.469764 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:41.470161 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:41.470162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:41.470162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:41.470221 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:41.470442 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:41.470443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:41.470443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:41.470554 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:41.470606 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:41.470903 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 02:13:41.471011 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:13:41.471181 23480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.462021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:41671 (local address 127.5.9.131:41033)
0809 02:13:41.462206 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:41.462252 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:41.462759 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:41.463015 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:41.463018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:41.463623 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:41.463826 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:41.463830 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.464601 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.464602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:41.465426 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:41.465428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.465983 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.465984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:41.466143 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.466791 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:13:41.466800 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:41.468260 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:41.469836 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:41.469843 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:41.469845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:41.470089 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:41.470283 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:41.470284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:41.470284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:41.470387 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:41.470609 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:41.470613 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:41.470738 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 02:13:41.470823 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 02:13:41.470973 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":104,"threads_started":1}
I20260809 02:13:41.471679 23477 heartbeater.cc:347] Connected to a master server at 127.5.9.190:41671
I20260809 02:13:41.471776 23477 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:41.471935 23477 heartbeater.cc:511] Master 127.5.9.190:41671 requested a full tablet report, sending...
I20260809 02:13:41.472353 22994 ts_manager.cc:194] Registered new tserver with Master: d2004d6845404f01a842cc9ded404f39 (127.5.9.131:37619)
I20260809 02:13:41.472700 22994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:41033
I20260809 02:13:41.478258 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:13:41.486127 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.479780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57840 (local address 127.5.9.190:41671)
0809 02:13:41.479879 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:41.479882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:41.479975 (+ 93us) server_negotiation.cc:407] Connection header received
0809 02:13:41.480059 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:41.480060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:41.480070 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:41.480128 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:41.480725 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.481234 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.482047 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.482251 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.482332 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.483540 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:41.483542 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:41.483542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:41.483550 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:41.484794 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:41.485179 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:41.485180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:41.485180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:41.485249 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:41.485455 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:41.485456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:41.485456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:41.485676 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:41.485793 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:41.485916 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 02:13:41.486006 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 02:13:41.487790 22994 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:13:41.489390 22994 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:13:41.496052 23493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.491947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45350 (local address 127.5.9.131:37619)
0809 02:13:41.492159 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:41.492163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:41.492366 (+ 203us) server_negotiation.cc:407] Connection header received
0809 02:13:41.492523 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:41.492526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:41.492544 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:41.492740 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:41.492986 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.493752 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.495092 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.495601 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.495688 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.495747 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:41.495921 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 02:13:41.495960 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":103,"threads_started":1}
I20260809 02:13:41.496039 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.491824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:37619 (local address 127.0.0.1:45350)
0809 02:13:41.492301 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:41.492310 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:41.492394 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:41.492835 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:41.492836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:41.492838 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:41.492888 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:41.492890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.493856 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.493857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:41.494999 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:41.495002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.495701 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.495702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:41.495802 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.495816 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 02:13:41.495907 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 02:13:41.495962 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":2,"thread_start_us":72,"threads_started":1}
I20260809 02:13:41.496080 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.491956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39300 (local address 127.5.9.130:37111)
0809 02:13:41.492238 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:41.492243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:41.492253 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:13:41.492395 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:41.492397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:41.492414 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:41.492501 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:41.492695 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.493571 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.494879 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.495624 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.495692 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.495747 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:41.495923 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 02:13:41.495963 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":85,"threads_started":1}
I20260809 02:13:41.496304 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.491824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:37111 (local address 127.0.0.1:39300)
0809 02:13:41.492144 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:41.492189 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:41.492264 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:41.492502 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:41.492505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:41.492507 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:41.492583 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:41.492585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.493713 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.493714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:41.494772 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:41.494775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.495781 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.495782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:41.495842 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.495846 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:13:41.496195 (+ 349us) client_negotiation.cc:240] Negotiation successful
0809 02:13:41.496240 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260809 02:13:41.497273 23269 tablet_service.cc:1467] Processing CreateTablet for tablet e58362d1a9aa43729ce03a57ef1a7d3e (DEFAULT_TABLE table=test-table [id=7d1d639eef234ab0b13331b629b051f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:41.497273 23411 tablet_service.cc:1467] Processing CreateTablet for tablet e58362d1a9aa43729ce03a57ef1a7d3e (DEFAULT_TABLE table=test-table [id=7d1d639eef234ab0b13331b629b051f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:41.497702 23411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e58362d1a9aa43729ce03a57ef1a7d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:41.497702 23269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e58362d1a9aa43729ce03a57ef1a7d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:41.497944 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.492051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:46533 (local address 127.0.0.1:33546)
0809 02:13:41.492597 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:41.492608 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:41.492655 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:41.492852 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:41.492853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:41.492855 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:41.492898 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:41.492900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.494954 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.494954 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:41.495988 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:41.495990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.497697 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.497699 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:41.497770 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.497775 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:13:41.497828 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 02:13:41.497880 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":79,"threads_started":1}
I20260809 02:13:41.498266 23495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.492187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33546 (local address 127.5.9.129:46533)
0809 02:13:41.492365 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:41.492369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:41.492716 (+ 347us) server_negotiation.cc:407] Connection header received
0809 02:13:41.492765 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:41.492768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:41.492784 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:41.492857 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:41.493700 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.494848 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.496079 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.496753 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.496825 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.498099 (+ 1274us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:41.498130 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 02:13:41.498180 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":103,"threads_started":1}
I20260809 02:13:41.499028 23125 tablet_service.cc:1467] Processing CreateTablet for tablet e58362d1a9aa43729ce03a57ef1a7d3e (DEFAULT_TABLE table=test-table [id=7d1d639eef234ab0b13331b629b051f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:41.499358 23125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e58362d1a9aa43729ce03a57ef1a7d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:41.501235 23496 tablet_bootstrap.cc:492] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39: Bootstrap starting.
I20260809 02:13:41.501435 23497 tablet_bootstrap.cc:492] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812: Bootstrap starting.
I20260809 02:13:41.502077 23497 tablet_bootstrap.cc:654] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:41.502351 23498 tablet_bootstrap.cc:492] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b: Bootstrap starting.
I20260809 02:13:41.502682 23497 log.cc:824] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:41.502985 23498 tablet_bootstrap.cc:654] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:41.503485 23496 tablet_bootstrap.cc:654] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:41.503751 23498 log.cc:824] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:41.504057 23496 log.cc:824] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:41.504580 23497 tablet_bootstrap.cc:492] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812: No bootstrap required, opened a new log
I20260809 02:13:41.504611 23498 tablet_bootstrap.cc:492] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b: No bootstrap required, opened a new log
I20260809 02:13:41.504662 23497 ts_tablet_manager.cc:1397] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:13:41.504691 23498 ts_tablet_manager.cc:1397] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:13:41.504860 23496 tablet_bootstrap.cc:492] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39: No bootstrap required, opened a new log
I20260809 02:13:41.504927 23496 ts_tablet_manager.cc:1397] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 02:13:41.506209 23498 raft_consensus.cc:348] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22a654a9bb049c4a2bfe07581e58812" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37111 } } peers { permanent_uuid: "d2004d6845404f01a842cc9ded404f39" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37619 } } peers { permanent_uuid: "3b1e56f0ef8f4a4eba243ee6802f343b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46533 } }
I20260809 02:13:41.506392 23498 raft_consensus.cc:374] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:41.506459 23498 raft_consensus.cc:729] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b1e56f0ef8f4a4eba243ee6802f343b, State: Initialized, Role: FOLLOWER
I20260809 02:13:41.506610 23498 consensus_queue.cc:260] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22a654a9bb049c4a2bfe07581e58812" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37111 } } peers { permanent_uuid: "d2004d6845404f01a842cc9ded404f39" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37619 } } peers { permanent_uuid: "3b1e56f0ef8f4a4eba243ee6802f343b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46533 } }
I20260809 02:13:41.506953 23498 ts_tablet_manager.cc:1428] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 02:13:41.506975 23196 heartbeater.cc:503] Master 127.5.9.190:41671 was elected leader, sending a full tablet report...
I20260809 02:13:41.506774 23497 raft_consensus.cc:348] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22a654a9bb049c4a2bfe07581e58812" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37111 } } peers { permanent_uuid: "d2004d6845404f01a842cc9ded404f39" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37619 } } peers { permanent_uuid: "3b1e56f0ef8f4a4eba243ee6802f343b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46533 } }
I20260809 02:13:41.506925 23496 raft_consensus.cc:348] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22a654a9bb049c4a2bfe07581e58812" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37111 } } peers { permanent_uuid: "d2004d6845404f01a842cc9ded404f39" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37619 } } peers { permanent_uuid: "3b1e56f0ef8f4a4eba243ee6802f343b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46533 } }
I20260809 02:13:41.507089 23496 raft_consensus.cc:374] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:41.507113 23497 raft_consensus.cc:374] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:41.507138 23496 raft_consensus.cc:729] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2004d6845404f01a842cc9ded404f39, State: Initialized, Role: FOLLOWER
I20260809 02:13:41.507192 23497 raft_consensus.cc:729] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a22a654a9bb049c4a2bfe07581e58812, State: Initialized, Role: FOLLOWER
I20260809 02:13:41.507257 23496 consensus_queue.cc:260] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22a654a9bb049c4a2bfe07581e58812" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37111 } } peers { permanent_uuid: "d2004d6845404f01a842cc9ded404f39" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37619 } } peers { permanent_uuid: "3b1e56f0ef8f4a4eba243ee6802f343b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46533 } }
I20260809 02:13:41.507676 23496 ts_tablet_manager.cc:1428] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 02:13:41.508536 23477 heartbeater.cc:503] Master 127.5.9.190:41671 was elected leader, sending a full tablet report...
I20260809 02:13:41.508761 23497 consensus_queue.cc:260] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22a654a9bb049c4a2bfe07581e58812" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37111 } } peers { permanent_uuid: "d2004d6845404f01a842cc9ded404f39" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37619 } } peers { permanent_uuid: "3b1e56f0ef8f4a4eba243ee6802f343b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46533 } }
I20260809 02:13:41.509056 23497 ts_tablet_manager.cc:1428] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 02:13:41.509212 23338 heartbeater.cc:503] Master 127.5.9.190:41671 was elected leader, sending a full tablet report...
W20260809 02:13:41.530467 23339 tablet.cc:2369] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:13:41.618939 23197 tablet.cc:2369] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:13:41.656004 23504 raft_consensus.cc:482] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:13:41.656136 23504 raft_consensus.cc:504] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22a654a9bb049c4a2bfe07581e58812" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37111 } } peers { permanent_uuid: "d2004d6845404f01a842cc9ded404f39" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37619 } } peers { permanent_uuid: "3b1e56f0ef8f4a4eba243ee6802f343b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46533 } }
I20260809 02:13:41.656543 23504 leader_election.cc:288] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2004d6845404f01a842cc9ded404f39 (127.5.9.131:37619), 3b1e56f0ef8f4a4eba243ee6802f343b (127.5.9.129:46533)
I20260809 02:13:41.659999 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.656781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:37619 (local address 127.5.9.130:43867)
0809 02:13:41.656843 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:41.656856 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:41.656906 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:41.657172 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:41.657173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:41.657175 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:41.657241 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:41.657242 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.658006 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.658007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:41.658839 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:41.658841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.659641 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.659642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:41.659781 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.659786 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:13:41.659862 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:13:41.659900 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 02:13:41.660009 23493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.656895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:43867 (local address 127.5.9.131:37619)
0809 02:13:41.656981 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:41.656983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:41.656995 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:13:41.657028 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:41.657029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:41.657040 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:41.657157 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:41.657332 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.657885 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.658929 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.659519 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.659606 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.659618 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:41.659885 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 02:13:41.659940 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 02:13:41.660094 23495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.657004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:40825 (local address 127.5.9.129:46533)
0809 02:13:41.657079 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:41.657082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:41.657233 (+ 151us) server_negotiation.cc:407] Connection header received
0809 02:13:41.657301 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:41.657303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:41.657315 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:41.657385 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:41.657550 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.658088 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.659098 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.659676 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.659759 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.659771 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:41.659967 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 02:13:41.660015 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:13:41.660125 23506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.656925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:46533 (local address 127.5.9.130:40825)
0809 02:13:41.657162 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:41.657174 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:41.657246 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:41.657392 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:41.657393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:41.657395 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:41.657474 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:41.657475 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.658183 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.658184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:41.658987 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:41.658989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.659782 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.659783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:41.659877 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.659881 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:13:41.659947 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 02:13:41.659987 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":163,"threads_started":1}
I20260809 02:13:41.660424 23148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e58362d1a9aa43729ce03a57ef1a7d3e" candidate_uuid: "a22a654a9bb049c4a2bfe07581e58812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b1e56f0ef8f4a4eba243ee6802f343b" is_pre_election: true
I20260809 02:13:41.660452 23431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e58362d1a9aa43729ce03a57ef1a7d3e" candidate_uuid: "a22a654a9bb049c4a2bfe07581e58812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2004d6845404f01a842cc9ded404f39" is_pre_election: true
I20260809 02:13:41.660594 23148 raft_consensus.cc:2454] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a22a654a9bb049c4a2bfe07581e58812 in term 0.
I20260809 02:13:41.660619 23431 raft_consensus.cc:2454] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a22a654a9bb049c4a2bfe07581e58812 in term 0.
I20260809 02:13:41.660840 23222 leader_election.cc:302] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b1e56f0ef8f4a4eba243ee6802f343b, a22a654a9bb049c4a2bfe07581e58812; no voters:
I20260809 02:13:41.661013 23504 raft_consensus.cc:2781] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:13:41.661087 23504 raft_consensus.cc:482] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:13:41.661106 23504 raft_consensus.cc:3037] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:41.661916 23504 raft_consensus.cc:504] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22a654a9bb049c4a2bfe07581e58812" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37111 } } peers { permanent_uuid: "d2004d6845404f01a842cc9ded404f39" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37619 } } peers { permanent_uuid: "3b1e56f0ef8f4a4eba243ee6802f343b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46533 } }
I20260809 02:13:41.662062 23504 leader_election.cc:288] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [CANDIDATE]: Term 1 election: Requested vote from peers d2004d6845404f01a842cc9ded404f39 (127.5.9.131:37619), 3b1e56f0ef8f4a4eba243ee6802f343b (127.5.9.129:46533)
I20260809 02:13:41.662236 23431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e58362d1a9aa43729ce03a57ef1a7d3e" candidate_uuid: "a22a654a9bb049c4a2bfe07581e58812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2004d6845404f01a842cc9ded404f39"
I20260809 02:13:41.662286 23148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e58362d1a9aa43729ce03a57ef1a7d3e" candidate_uuid: "a22a654a9bb049c4a2bfe07581e58812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b1e56f0ef8f4a4eba243ee6802f343b"
I20260809 02:13:41.662346 23431 raft_consensus.cc:3037] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:41.662379 23148 raft_consensus.cc:3037] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:41.663072 23431 raft_consensus.cc:2454] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a22a654a9bb049c4a2bfe07581e58812 in term 1.
I20260809 02:13:41.663098 23148 raft_consensus.cc:2454] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a22a654a9bb049c4a2bfe07581e58812 in term 1.
I20260809 02:13:41.663285 23222 leader_election.cc:302] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a22a654a9bb049c4a2bfe07581e58812, d2004d6845404f01a842cc9ded404f39; no voters:
I20260809 02:13:41.663384 23504 raft_consensus.cc:2781] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:41.663530 23504 raft_consensus.cc:686] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [term 1 LEADER]: Becoming Leader. State: Replica: a22a654a9bb049c4a2bfe07581e58812, State: Running, Role: LEADER
I20260809 02:13:41.663604 23504 consensus_queue.cc:237] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22a654a9bb049c4a2bfe07581e58812" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37111 } } peers { permanent_uuid: "d2004d6845404f01a842cc9ded404f39" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37619 } } peers { permanent_uuid: "3b1e56f0ef8f4a4eba243ee6802f343b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46533 } }
I20260809 02:13:41.664518 22993 catalog_manager.cc:5495] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 reported cstate change: term changed from 0 to 1, leader changed from <none> to a22a654a9bb049c4a2bfe07581e58812 (127.5.9.130). New cstate: current_term: 1 leader_uuid: "a22a654a9bb049c4a2bfe07581e58812" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22a654a9bb049c4a2bfe07581e58812" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2004d6845404f01a842cc9ded404f39" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b1e56f0ef8f4a4eba243ee6802f343b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46533 } health_report { overall_health: UNKNOWN } } }
I20260809 02:13:41.677500 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.675295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39302 (local address 127.5.9.130:37111)
0809 02:13:41.675362 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:41.675366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:41.675379 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:13:41.675414 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:41.675415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:41.675433 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:41.675503 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:13:41.675635 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.676117 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.676838 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.677032 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.677085 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.677218 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:13:41.677318 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:13:41.677379 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:41.677419 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 02:13:41.677438 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:13:41.679510 23148 raft_consensus.cc:1261] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b [term 1 FOLLOWER]: Refusing update from remote peer a22a654a9bb049c4a2bfe07581e58812: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:13:41.679514 23431 raft_consensus.cc:1261] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39 [term 1 FOLLOWER]: Refusing update from remote peer a22a654a9bb049c4a2bfe07581e58812: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:13:41.679935 23504 consensus_queue.cc:1035] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b1e56f0ef8f4a4eba243ee6802f343b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:13:41.680111 23507 consensus_queue.cc:1035] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2004d6845404f01a842cc9ded404f39" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:13:41.685591 22994 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:13:41.685745 22994 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:13:41.686645 22994 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b602adb65b143b49093ed400e25d9a9: Sending DeleteTablet for 3 replicas of tablet e58362d1a9aa43729ce03a57ef1a7d3e
I20260809 02:13:41.687079 23125 tablet_service.cc:1514] Processing DeleteTablet for tablet e58362d1a9aa43729ce03a57ef1a7d3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:41 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:33546
I20260809 02:13:41.687137 23411 tablet_service.cc:1514] Processing DeleteTablet for tablet e58362d1a9aa43729ce03a57ef1a7d3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:41 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:45350
I20260809 02:13:41.687117 23269 tablet_service.cc:1514] Processing DeleteTablet for tablet e58362d1a9aa43729ce03a57ef1a7d3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:41 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:39300
I20260809 02:13:41.687598 23523 tablet_replica.cc:330] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b: stopping tablet replica
I20260809 02:13:41.687603 23522 tablet_replica.cc:330] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812: stopping tablet replica
I20260809 02:13:41.687703 23522 raft_consensus.cc:2229] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:13:41.687738 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 23059
I20260809 02:13:41.687743 23523 raft_consensus.cc:2229] T e58362d1a9aa43729ce03a57ef1a7d3e P 3b1e56f0ef8f4a4eba243ee6802f343b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:13:41.687866 23522 raft_consensus.cc:2258] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:13:41.688028 23521 tablet_replica.cc:330] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39: stopping tablet replica
I20260809 02:13:41.688126 23521 raft_consensus.cc:2229] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:13:41.688284 23521 raft_consensus.cc:2258] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:13:41.689148 23521 ts_tablet_manager.cc:1905] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:13:41.690944 23522 ts_tablet_manager.cc:1905] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:13:41.691324 23521 ts_tablet_manager.cc:1918] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:13:41.691429 23521 log.cc:1196] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/e58362d1a9aa43729ce03a57ef1a7d3e
I20260809 02:13:41.691754 23521 ts_tablet_manager.cc:1939] T e58362d1a9aa43729ce03a57ef1a7d3e P d2004d6845404f01a842cc9ded404f39: Deleting consensus metadata
I20260809 02:13:41.692324 22981 catalog_manager.cc:4841] TS d2004d6845404f01a842cc9ded404f39 (127.5.9.131:37619): tablet e58362d1a9aa43729ce03a57ef1a7d3e (table test-table [id=7d1d639eef234ab0b13331b629b051f5]) successfully deleted
I20260809 02:13:41.692901 23522 ts_tablet_manager.cc:1918] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:13:41.692968 23522 log.cc:1196] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/e58362d1a9aa43729ce03a57ef1a7d3e
I20260809 02:13:41.693281 23522 ts_tablet_manager.cc:1939] T e58362d1a9aa43729ce03a57ef1a7d3e P a22a654a9bb049c4a2bfe07581e58812: Deleting consensus metadata
I20260809 02:13:41.693744 22978 catalog_manager.cc:4841] TS a22a654a9bb049c4a2bfe07581e58812 (127.5.9.130:37111): tablet e58362d1a9aa43729ce03a57ef1a7d3e (table test-table [id=7d1d639eef234ab0b13331b629b051f5]) successfully deleted
W20260809 02:13:41.697526 22981 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:46533 (error 108)
I20260809 02:13:41.698276 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.698109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:46533 (local address 127.0.0.1:33554)
0809 02:13:41.698181 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:41.698233 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:46533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 02:13:41.698287 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 23201
W20260809 02:13:41.699785 22981 catalog_manager.cc:4584] TS 3b1e56f0ef8f4a4eba243ee6802f343b (127.5.9.129:46533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e58362d1a9aa43729ce03a57ef1a7d3e: Network error: Client connection negotiation failed: client connection to 127.5.9.129:46533: connect: Connection refused (error 111)
I20260809 02:13:41.705587 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 23343
I20260809 02:13:41.711611 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 22961
2026-08-09T02:13:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2172 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T02:13:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:13:41Z Disabled control of system clock
I20260809 02:13:41.743072 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:42373
--webserver_interface=127.5.9.190
--webserver_port=0
--builtin_ntp_servers=127.5.9.148:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:42373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:13:41.828940 23529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:41.829195 23529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:41.829216 23529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:41.830559 23529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:13:41.830614 23529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:41.830627 23529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:41.830679 23529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:13:41.830705 23529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:13:41.832181 23529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:42373
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:42373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:41.832485 23529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:41.832837 23529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:41.838042 23537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:41.838097 23535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:41.838043 23534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:41.838101 23529 server_base.cc:1034] running on GCE node
I20260809 02:13:41.838478 23529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:41.838740 23529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:41.839892 23529 hybrid_clock.cc:648] HybridClock initialized: now 1786241621839868 us; error 47 us; skew 500 ppm
I20260809 02:13:41.841218 23529 webserver.cc:460] Webserver started at http://127.5.9.190:38161/ using document root <none> and password file <none>
I20260809 02:13:41.841451 23529 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:41.841498 23529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:41.841604 23529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:41.842773 23529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "0e0bbfd0c2ad49edae5a5b6a6e4b4038"
format_stamp: "Formatted at 2026-08-09 02:13:41 on dist-test-slave-p3t5"
server_key: "f15e14539d105c03c144af368a6c0c01"
server_key_iv: "288704927d8c5481e2626447abcd96e1"
server_key_version: "encryptionkey@0"
I20260809 02:13:41.843107 23529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "0e0bbfd0c2ad49edae5a5b6a6e4b4038"
format_stamp: "Formatted at 2026-08-09 02:13:41 on dist-test-slave-p3t5"
server_key: "f15e14539d105c03c144af368a6c0c01"
server_key_iv: "288704927d8c5481e2626447abcd96e1"
server_key_version: "encryptionkey@0"
I20260809 02:13:41.844465 23529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:13:41.845254 23543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:41.845443 23529 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:13:41.845546 23529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "0e0bbfd0c2ad49edae5a5b6a6e4b4038"
format_stamp: "Formatted at 2026-08-09 02:13:41 on dist-test-slave-p3t5"
server_key: "f15e14539d105c03c144af368a6c0c01"
server_key_iv: "288704927d8c5481e2626447abcd96e1"
server_key_version: "encryptionkey@0"
I20260809 02:13:41.845654 23529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:41.868772 23529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:41.869108 23529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:41.869225 23529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:41.873224 23529 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:42373
I20260809 02:13:41.873337 23599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:42373 every 8 connection(s)
I20260809 02:13:41.873729 23529 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:13:41.874826 23600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:41.877202 23600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038: Bootstrap starting.
I20260809 02:13:41.877897 23600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:41.878221 23600 log.cc:824] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:41.878824 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 23529
I20260809 02:13:41.878957 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:13:41.879099 23600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038: No bootstrap required, opened a new log
I20260809 02:13:41.879197 5158 external_mini_cluster.cc:1428] Setting key db743e79b73a7629eb6e851ca046262b
I20260809 02:13:41.881035 23600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0bbfd0c2ad49edae5a5b6a6e4b4038" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42373 } }
I20260809 02:13:41.881209 23600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:41.881275 23600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0bbfd0c2ad49edae5a5b6a6e4b4038, State: Initialized, Role: FOLLOWER
I20260809 02:13:41.881474 23600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0bbfd0c2ad49edae5a5b6a6e4b4038" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42373 } }
I20260809 02:13:41.881561 23600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:13:41.881585 23600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:13:41.881659 23600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:41.882252 23600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0bbfd0c2ad49edae5a5b6a6e4b4038" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42373 } }
I20260809 02:13:41.882351 23600 leader_election.cc:302] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e0bbfd0c2ad49edae5a5b6a6e4b4038; no voters:
I20260809 02:13:41.882586 23600 leader_election.cc:288] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:13:41.882635 23605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:41.882771 23605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [term 1 LEADER]: Becoming Leader. State: Replica: 0e0bbfd0c2ad49edae5a5b6a6e4b4038, State: Running, Role: LEADER
I20260809 02:13:41.882872 23600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:13:41.882923 23605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0bbfd0c2ad49edae5a5b6a6e4b4038" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42373 } }
I20260809 02:13:41.883400 23605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e0bbfd0c2ad49edae5a5b6a6e4b4038. Latest consensus state: current_term: 1 leader_uuid: "0e0bbfd0c2ad49edae5a5b6a6e4b4038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0bbfd0c2ad49edae5a5b6a6e4b4038" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42373 } } }
I20260809 02:13:41.883664 23605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:41.883589 23607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e0bbfd0c2ad49edae5a5b6a6e4b4038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e0bbfd0c2ad49edae5a5b6a6e4b4038" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42373 } } }
I20260809 02:13:41.884048 23607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:41.884328 23614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:13:41.885200 23614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:13:41.887115 23604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:41.879776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49894 (local address 127.5.9.190:42373)
0809 02:13:41.880043 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:41.880051 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:41.880077 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:13:41.881012 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:41.881018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:41.881078 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:41.881225 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:41.881471 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.883310 (+ 1839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.884581 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:41.885212 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:41.885365 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:41.886206 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:41.886210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:41.886210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:41.886446 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:41.886499 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:41.886511 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:41.886592 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:41.886693 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 02:13:41.886901 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":122,"threads_started":1}
I20260809 02:13:41.887748 23614 catalog_manager.cc:1347] Generated new cluster ID: cf34c4433f524b43912eaa57227875de
I20260809 02:13:41.887848 23614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:13:41.906657 23614 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:13:41.907230 23614 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:13:41.912686 23614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038: Generated new TSK 0
I20260809 02:13:41.912914 23614 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:13:41.921406 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42373
--builtin_ntp_servers=127.5.9.148:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:13:42.009868 23624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:42.010097 23624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:42.010121 23624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:42.011339 23624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:42.011389 23624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:42.011415 23624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:13:42.012825 23624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:42.013078 23624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:42.013425 23624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:42.019187 23633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:42.019238 23631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:42.019201 23630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:42.019445 23624 server_base.cc:1034] running on GCE node
I20260809 02:13:42.019691 23624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:42.020004 23624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:42.021193 23624 hybrid_clock.cc:648] HybridClock initialized: now 1786241622021185 us; error 40 us; skew 500 ppm
I20260809 02:13:42.022763 23624 webserver.cc:460] Webserver started at http://127.5.9.129:42191/ using document root <none> and password file <none>
I20260809 02:13:42.023054 23624 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:42.023132 23624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:42.023293 23624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:42.024487 23624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "eb82ef3d2ac047e4968fb7599e581258"
format_stamp: "Formatted at 2026-08-09 02:13:42 on dist-test-slave-p3t5"
server_key: "83398ba5d70381f2e1f2745844a673bc"
server_key_iv: "745206d10450dac7175209cf643f402f"
server_key_version: "encryptionkey@0"
I20260809 02:13:42.024881 23624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "eb82ef3d2ac047e4968fb7599e581258"
format_stamp: "Formatted at 2026-08-09 02:13:42 on dist-test-slave-p3t5"
server_key: "83398ba5d70381f2e1f2745844a673bc"
server_key_iv: "745206d10450dac7175209cf643f402f"
server_key_version: "encryptionkey@0"
I20260809 02:13:42.026486 23624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:13:42.027361 23639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:42.027559 23624 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:42.027678 23624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "eb82ef3d2ac047e4968fb7599e581258"
format_stamp: "Formatted at 2026-08-09 02:13:42 on dist-test-slave-p3t5"
server_key: "83398ba5d70381f2e1f2745844a673bc"
server_key_iv: "745206d10450dac7175209cf643f402f"
server_key_version: "encryptionkey@0"
I20260809 02:13:42.027761 23624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:42.055727 23624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:42.056114 23624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:42.056283 23624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:42.056545 23624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:42.056917 23624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:42.056975 23624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:42.057022 23624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:42.057054 23624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:42.063447 23624 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:32921
I20260809 02:13:42.063499 23758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:32921 every 8 connection(s)
I20260809 02:13:42.063850 23624 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:13:42.068244 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 23624
I20260809 02:13:42.068394 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:13:42.068634 5158 external_mini_cluster.cc:1428] Setting key a913a18ffd29abd8cbd85e726e8c5996
I20260809 02:13:42.069585 23604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.064624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:56099 (local address 127.5.9.190:42373)
0809 02:13:42.064727 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:42.064730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:42.064937 (+ 207us) server_negotiation.cc:407] Connection header received
0809 02:13:42.065429 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:42.065430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:42.065442 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:42.065497 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:42.066255 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.066780 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.067782 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.067968 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.068019 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.069245 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:42.069247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:42.069247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:42.069253 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:42.069272 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:42.069276 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:42.069351 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:42.069456 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 02:13:42.069492 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 02:13:42.069881 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42373
--builtin_ntp_servers=127.5.9.148:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:13:42.069893 23761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.064583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:42373 (local address 127.5.9.129:56099)
0809 02:13:42.064804 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:42.064840 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:42.065326 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:42.065550 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:42.065554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:42.065562 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:42.066159 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:42.066164 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.066871 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.066872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:42.067706 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:42.067708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.068081 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.068082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:42.068249 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.069075 (+ 826us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:13:42.069076 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:42.069114 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:13:42.069149 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:42.069371 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:42.069374 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:42.069378 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:13:42.069460 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 02:13:42.069590 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":111,"threads_started":1}
I20260809 02:13:42.070588 23759 heartbeater.cc:347] Connected to a master server at 127.5.9.190:42373
I20260809 02:13:42.070698 23759 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:42.070902 23759 heartbeater.cc:511] Master 127.5.9.190:42373 requested a full tablet report, sending...
I20260809 02:13:42.071420 23562 ts_manager.cc:194] Registered new tserver with Master: eb82ef3d2ac047e4968fb7599e581258 (127.5.9.129:32921)
I20260809 02:13:42.072435 23562 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.9.129:56099
W20260809 02:13:42.156533 23762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:42.156770 23762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:42.156795 23762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:42.158025 23762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:42.158072 23762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:42.158098 23762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:13:42.159459 23762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:42.159688 23762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:42.159970 23762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:42.165266 23767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:42.165285 23770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:42.165266 23768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:42.165673 23762 server_base.cc:1034] running on GCE node
I20260809 02:13:42.165905 23762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:42.166116 23762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:42.167256 23762 hybrid_clock.cc:648] HybridClock initialized: now 1786241622167245 us; error 27 us; skew 500 ppm
I20260809 02:13:42.168609 23762 webserver.cc:460] Webserver started at http://127.5.9.130:43595/ using document root <none> and password file <none>
I20260809 02:13:42.168913 23762 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:42.168967 23762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:42.169111 23762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:42.170321 23762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "79ac667bbcc54234819d8f7f9a8d2117"
format_stamp: "Formatted at 2026-08-09 02:13:42 on dist-test-slave-p3t5"
server_key: "1f8a9325bec54d99a54c0296be1172ce"
server_key_iv: "598e6a3ffc88bc5558dc3662f734060a"
server_key_version: "encryptionkey@0"
I20260809 02:13:42.170701 23762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "79ac667bbcc54234819d8f7f9a8d2117"
format_stamp: "Formatted at 2026-08-09 02:13:42 on dist-test-slave-p3t5"
server_key: "1f8a9325bec54d99a54c0296be1172ce"
server_key_iv: "598e6a3ffc88bc5558dc3662f734060a"
server_key_version: "encryptionkey@0"
I20260809 02:13:42.172098 23762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:13:42.172878 23776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:42.173082 23762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:42.173179 23762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "79ac667bbcc54234819d8f7f9a8d2117"
format_stamp: "Formatted at 2026-08-09 02:13:42 on dist-test-slave-p3t5"
server_key: "1f8a9325bec54d99a54c0296be1172ce"
server_key_iv: "598e6a3ffc88bc5558dc3662f734060a"
server_key_version: "encryptionkey@0"
I20260809 02:13:42.173257 23762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:42.192561 23762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:42.192940 23762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:42.193105 23762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:42.193364 23762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:42.193794 23762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:42.193832 23762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:42.193881 23762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:42.193908 23762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:42.200026 23762 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:33837
I20260809 02:13:42.200071 23900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:33837 every 8 connection(s)
I20260809 02:13:42.200433 23762 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:13:42.205863 23604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.201234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:48435 (local address 127.5.9.190:42373)
0809 02:13:42.201328 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:42.201330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:42.201554 (+ 224us) server_negotiation.cc:407] Connection header received
0809 02:13:42.202073 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:42.202074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:42.202084 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:42.202139 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:42.202821 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.203333 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.204346 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.204544 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.204598 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.205556 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:42.205557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:42.205557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:42.205561 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:42.205574 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:42.205577 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:42.205691 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:42.205761 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 02:13:42.205789 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:13:42.206091 23903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.201188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:42373 (local address 127.5.9.130:48435)
0809 02:13:42.201415 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:42.201467 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:42.201980 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:42.202196 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:42.202199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:42.202208 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:42.202745 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:42.202748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.203431 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.203431 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:42.204258 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:42.204260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.204627 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.204628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:42.204748 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.205405 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:13:42.205405 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:42.205443 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:13:42.205469 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:42.205692 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:42.205695 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:42.205697 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:13:42.205747 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 02:13:42.205834 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":110,"threads_started":1}
I20260809 02:13:42.206362 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 23762
I20260809 02:13:42.206511 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:13:42.206769 5158 external_mini_cluster.cc:1428] Setting key 35a0b90f94ef67b38f6628bc943b58e4
I20260809 02:13:42.206828 23901 heartbeater.cc:347] Connected to a master server at 127.5.9.190:42373
I20260809 02:13:42.206938 23901 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:42.207151 23901 heartbeater.cc:511] Master 127.5.9.190:42373 requested a full tablet report, sending...
I20260809 02:13:42.207593 23562 ts_manager.cc:194] Registered new tserver with Master: 79ac667bbcc54234819d8f7f9a8d2117 (127.5.9.130:33837)
I20260809 02:13:42.207870 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42373
--builtin_ntp_servers=127.5.9.148:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:13:42.208063 23562 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.9.130:48435
W20260809 02:13:42.293484 23904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:42.293732 23904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:42.293756 23904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:42.295023 23904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:42.295078 23904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:42.295105 23904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:13:42.296528 23904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:42.296810 23904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:42.297134 23904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:42.302405 23909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:42.302482 23912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:42.302405 23910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:42.302836 23904 server_base.cc:1034] running on GCE node
I20260809 02:13:42.303087 23904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:42.303400 23904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:42.304594 23904 hybrid_clock.cc:648] HybridClock initialized: now 1786241622304569 us; error 54 us; skew 500 ppm
I20260809 02:13:42.306262 23904 webserver.cc:460] Webserver started at http://127.5.9.131:46555/ using document root <none> and password file <none>
I20260809 02:13:42.306566 23904 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:42.306648 23904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:42.306833 23904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:42.308128 23904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "ba4ef8041a214ee78242297040c5d7e2"
format_stamp: "Formatted at 2026-08-09 02:13:42 on dist-test-slave-p3t5"
server_key: "cee5286bba9288c314b31a11364e09a4"
server_key_iv: "8f0b0ae34172f696bd0227ce79d502de"
server_key_version: "encryptionkey@0"
I20260809 02:13:42.308645 23904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "ba4ef8041a214ee78242297040c5d7e2"
format_stamp: "Formatted at 2026-08-09 02:13:42 on dist-test-slave-p3t5"
server_key: "cee5286bba9288c314b31a11364e09a4"
server_key_iv: "8f0b0ae34172f696bd0227ce79d502de"
server_key_version: "encryptionkey@0"
I20260809 02:13:42.310221 23904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:13:42.311115 23918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:42.311329 23904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:13:42.311426 23904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "ba4ef8041a214ee78242297040c5d7e2"
format_stamp: "Formatted at 2026-08-09 02:13:42 on dist-test-slave-p3t5"
server_key: "cee5286bba9288c314b31a11364e09a4"
server_key_iv: "8f0b0ae34172f696bd0227ce79d502de"
server_key_version: "encryptionkey@0"
I20260809 02:13:42.311489 23904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:42.334107 23904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:42.334417 23904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:42.334539 23904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:42.334797 23904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:42.335152 23904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:42.335192 23904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:42.335217 23904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:42.335230 23904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:42.341261 23904 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:35973
I20260809 02:13:42.341331 24042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:35973 every 8 connection(s)
I20260809 02:13:42.341720 23904 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:13:42.344147 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 23904
I20260809 02:13:42.344270 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:13:42.344470 5158 external_mini_cluster.cc:1428] Setting key e4cf024190b8a2e93e99303b1c64238e
I20260809 02:13:42.347632 23604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.342624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:50267 (local address 127.5.9.190:42373)
0809 02:13:42.342701 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:42.342704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:42.343004 (+ 300us) server_negotiation.cc:407] Connection header received
0809 02:13:42.343498 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:42.343499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:42.343509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:42.343567 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:42.344335 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.345021 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.346218 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.346412 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.346473 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.347302 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:42.347304 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:42.347304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:42.347308 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:42.347323 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:42.347327 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:42.347408 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:42.347540 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 02:13:42.347569 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:13:42.347954 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.342620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:42373 (local address 127.5.9.131:50267)
0809 02:13:42.342868 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:42.342906 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:42.343409 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:42.343661 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:42.343664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:42.343673 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:42.344201 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:42.344205 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.345116 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.345117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:42.346137 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:42.346140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.346490 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.346491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:42.346606 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.347156 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:13:42.347157 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:42.347188 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:13:42.347211 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:42.347439 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:42.347446 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:42.347449 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:13:42.347522 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 02:13:42.347621 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":143,"threads_started":1}
I20260809 02:13:42.348510 24043 heartbeater.cc:347] Connected to a master server at 127.5.9.190:42373
I20260809 02:13:42.348620 24043 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:42.348775 24043 heartbeater.cc:511] Master 127.5.9.190:42373 requested a full tablet report, sending...
I20260809 02:13:42.349164 23562 ts_manager.cc:194] Registered new tserver with Master: ba4ef8041a214ee78242297040c5d7e2 (127.5.9.131:35973)
I20260809 02:13:42.349538 23562 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.9.131:50267
I20260809 02:13:42.356473 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:13:42.361137 23604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.358083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49906 (local address 127.5.9.190:42373)
0809 02:13:42.358152 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:42.358154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:42.358271 (+ 117us) server_negotiation.cc:407] Connection header received
0809 02:13:42.358345 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:42.358346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:42.358359 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:42.358416 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:42.358556 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.359057 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.359824 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.360019 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.360089 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.360868 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:42.360870 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:42.360870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:42.360874 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:42.360889 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:42.360893 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:42.360940 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:42.361042 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 02:13:42.361070 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:13:42.362970 23562 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:13:42.364569 23562 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:13:42.372089 24056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.367516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58218 (local address 127.5.9.130:33837)
0809 02:13:42.368036 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:42.368041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:42.368052 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:13:42.368097 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:42.368099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:42.368113 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:42.368208 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:42.368327 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.369116 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.370660 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.371540 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.371677 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.371800 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:42.371926 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 02:13:42.371976 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":128,"threads_started":1}
I20260809 02:13:42.372107 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.367403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:33837 (local address 127.0.0.1:58218)
0809 02:13:42.367845 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:42.367854 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:42.367895 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:42.368218 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:42.368219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:42.368221 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:42.368271 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:42.368273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.369207 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.369208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:42.370568 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:42.370571 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.371703 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.371704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:42.371791 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.371809 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 02:13:42.371890 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 02:13:42.371998 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":49,"thread_start_us":89,"threads_started":1}
I20260809 02:13:42.372117 24052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.367162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:32921 (local address 127.0.0.1:57048)
0809 02:13:42.367535 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:42.367579 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:42.367713 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:42.367981 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:42.367986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:42.367990 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:42.368078 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:42.368081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.369710 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.369712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:42.370758 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:42.370760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.371741 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.371742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:42.371816 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.371821 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:13:42.371890 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 02:13:42.371992 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":3,"thread_start_us":71,"threads_started":1}
I20260809 02:13:42.372599 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.367303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57048 (local address 127.5.9.129:32921)
0809 02:13:42.367591 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:42.367599 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:42.367686 (+ 87us) server_negotiation.cc:407] Connection header received
0809 02:13:42.367817 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:42.367820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:42.367851 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:42.368169 (+ 318us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:42.368240 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.369571 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.370847 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.371638 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.372299 (+ 661us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.372438 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:42.372468 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 02:13:42.372511 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":112,"threads_started":1}
I20260809 02:13:42.372790 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.367167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:35973 (local address 127.0.0.1:48888)
0809 02:13:42.367747 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:42.367759 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:42.367830 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:42.368110 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:42.368110 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:42.368112 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:42.368158 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:42.368160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.370832 (+ 2672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.370832 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:42.371479 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:42.371480 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.372559 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.372560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:42.372620 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.372624 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:13:42.372675 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 02:13:42.372723 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":93,"threads_started":1}
I20260809 02:13:42.372831 24053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.367261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48888 (local address 127.5.9.131:35973)
0809 02:13:42.367729 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:42.367733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:42.367900 (+ 167us) server_negotiation.cc:407] Connection header received
0809 02:13:42.367947 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:42.367950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:42.367966 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:42.368039 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:42.368241 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.369570 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.371575 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.372465 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.372555 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.372643 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:42.372703 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 02:13:42.372749 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":116,"threads_started":1}
I20260809 02:13:42.373474 23691 tablet_service.cc:1467] Processing CreateTablet for tablet b4d5a738fbc541d7afd1cad008be600c (DEFAULT_TABLE table=test-table [id=920a4bb2ec5f4690983990ac7d223245]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:42.373929 23691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4d5a738fbc541d7afd1cad008be600c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:42.373901 23973 tablet_service.cc:1467] Processing CreateTablet for tablet b4d5a738fbc541d7afd1cad008be600c (DEFAULT_TABLE table=test-table [id=920a4bb2ec5f4690983990ac7d223245]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:42.373901 23829 tablet_service.cc:1467] Processing CreateTablet for tablet b4d5a738fbc541d7afd1cad008be600c (DEFAULT_TABLE table=test-table [id=920a4bb2ec5f4690983990ac7d223245]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:42.374272 23973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4d5a738fbc541d7afd1cad008be600c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:42.374272 23829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4d5a738fbc541d7afd1cad008be600c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:42.376987 24057 tablet_bootstrap.cc:492] T b4d5a738fbc541d7afd1cad008be600c P eb82ef3d2ac047e4968fb7599e581258: Bootstrap starting.
I20260809 02:13:42.377699 24057 tablet_bootstrap.cc:654] T b4d5a738fbc541d7afd1cad008be600c P eb82ef3d2ac047e4968fb7599e581258: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:42.378017 24058 tablet_bootstrap.cc:492] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117: Bootstrap starting.
I20260809 02:13:42.378073 24057 log.cc:824] T b4d5a738fbc541d7afd1cad008be600c P eb82ef3d2ac047e4968fb7599e581258: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:42.378532 24059 tablet_bootstrap.cc:492] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2: Bootstrap starting.
I20260809 02:13:42.378775 24058 tablet_bootstrap.cc:654] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:42.379137 24058 log.cc:824] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:42.379240 24059 tablet_bootstrap.cc:654] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:42.379339 24057 tablet_bootstrap.cc:492] T b4d5a738fbc541d7afd1cad008be600c P eb82ef3d2ac047e4968fb7599e581258: No bootstrap required, opened a new log
I20260809 02:13:42.379455 24057 ts_tablet_manager.cc:1397] T b4d5a738fbc541d7afd1cad008be600c P eb82ef3d2ac047e4968fb7599e581258: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 02:13:42.379624 24059 log.cc:824] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:42.380131 24058 tablet_bootstrap.cc:492] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117: No bootstrap required, opened a new log
I20260809 02:13:42.380282 24058 ts_tablet_manager.cc:1397] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:13:42.380402 24059 tablet_bootstrap.cc:492] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2: No bootstrap required, opened a new log
I20260809 02:13:42.380497 24059 ts_tablet_manager.cc:1397] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:13:42.382347 24057 raft_consensus.cc:348] T b4d5a738fbc541d7afd1cad008be600c P eb82ef3d2ac047e4968fb7599e581258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb82ef3d2ac047e4968fb7599e581258" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 32921 } } peers { permanent_uuid: "ba4ef8041a214ee78242297040c5d7e2" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35973 } } peers { permanent_uuid: "79ac667bbcc54234819d8f7f9a8d2117" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33837 } }
I20260809 02:13:42.382541 24057 raft_consensus.cc:374] T b4d5a738fbc541d7afd1cad008be600c P eb82ef3d2ac047e4968fb7599e581258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:42.382587 24057 raft_consensus.cc:729] T b4d5a738fbc541d7afd1cad008be600c P eb82ef3d2ac047e4968fb7599e581258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb82ef3d2ac047e4968fb7599e581258, State: Initialized, Role: FOLLOWER
I20260809 02:13:42.382709 24057 consensus_queue.cc:260] T b4d5a738fbc541d7afd1cad008be600c P eb82ef3d2ac047e4968fb7599e581258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb82ef3d2ac047e4968fb7599e581258" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 32921 } } peers { permanent_uuid: "ba4ef8041a214ee78242297040c5d7e2" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35973 } } peers { permanent_uuid: "79ac667bbcc54234819d8f7f9a8d2117" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33837 } }
I20260809 02:13:42.382998 24057 ts_tablet_manager.cc:1428] T b4d5a738fbc541d7afd1cad008be600c P eb82ef3d2ac047e4968fb7599e581258: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260809 02:13:42.383137 23759 heartbeater.cc:503] Master 127.5.9.190:42373 was elected leader, sending a full tablet report...
I20260809 02:13:42.383068 24058 raft_consensus.cc:348] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb82ef3d2ac047e4968fb7599e581258" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 32921 } } peers { permanent_uuid: "ba4ef8041a214ee78242297040c5d7e2" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35973 } } peers { permanent_uuid: "79ac667bbcc54234819d8f7f9a8d2117" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33837 } }
I20260809 02:13:42.383109 24059 raft_consensus.cc:348] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb82ef3d2ac047e4968fb7599e581258" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 32921 } } peers { permanent_uuid: "ba4ef8041a214ee78242297040c5d7e2" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35973 } } peers { permanent_uuid: "79ac667bbcc54234819d8f7f9a8d2117" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33837 } }
I20260809 02:13:42.383268 24058 raft_consensus.cc:374] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:42.383307 24058 raft_consensus.cc:729] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ac667bbcc54234819d8f7f9a8d2117, State: Initialized, Role: FOLLOWER
I20260809 02:13:42.383379 24059 raft_consensus.cc:374] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:42.383482 24059 raft_consensus.cc:729] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba4ef8041a214ee78242297040c5d7e2, State: Initialized, Role: FOLLOWER
I20260809 02:13:42.383504 24058 consensus_queue.cc:260] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb82ef3d2ac047e4968fb7599e581258" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 32921 } } peers { permanent_uuid: "ba4ef8041a214ee78242297040c5d7e2" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35973 } } peers { permanent_uuid: "79ac667bbcc54234819d8f7f9a8d2117" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33837 } }
I20260809 02:13:42.383623 24059 consensus_queue.cc:260] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb82ef3d2ac047e4968fb7599e581258" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 32921 } } peers { permanent_uuid: "ba4ef8041a214ee78242297040c5d7e2" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35973 } } peers { permanent_uuid: "79ac667bbcc54234819d8f7f9a8d2117" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33837 } }
I20260809 02:13:42.383899 24058 ts_tablet_manager.cc:1428] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 02:13:42.384008 24043 heartbeater.cc:503] Master 127.5.9.190:42373 was elected leader, sending a full tablet report...
I20260809 02:13:42.384093 23901 heartbeater.cc:503] Master 127.5.9.190:42373 was elected leader, sending a full tablet report...
I20260809 02:13:42.384269 24059 ts_tablet_manager.cc:1428] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 02:13:42.427189 24065 raft_consensus.cc:482] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:13:42.427340 24065 raft_consensus.cc:504] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb82ef3d2ac047e4968fb7599e581258" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 32921 } } peers { permanent_uuid: "ba4ef8041a214ee78242297040c5d7e2" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35973 } } peers { permanent_uuid: "79ac667bbcc54234819d8f7f9a8d2117" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33837 } }
I20260809 02:13:42.427661 24065 leader_election.cc:288] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb82ef3d2ac047e4968fb7599e581258 (127.5.9.129:32921), 79ac667bbcc54234819d8f7f9a8d2117 (127.5.9.130:33837)
I20260809 02:13:42.431129 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.427893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:32921 (local address 127.5.9.131:50291)
0809 02:13:42.427959 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:42.427973 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:42.428048 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:42.428250 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:42.428252 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:42.428253 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:42.428335 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:42.428337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.429185 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.429186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:42.430117 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:42.430120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.430851 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.430852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:42.430938 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.430943 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:13:42.431001 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 02:13:42.431039 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":30}
I20260809 02:13:42.431294 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.428035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:50291 (local address 127.5.9.129:32921)
0809 02:13:42.428090 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:42.428092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:42.428102 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:13:42.428136 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:42.428137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:42.428149 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:42.428222 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:42.428426 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.429104 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.430236 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.430665 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.430722 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.430731 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:42.431136 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 02:13:42.431208 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 02:13:42.431463 24066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.428006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:33837 (local address 127.5.9.131:57379)
0809 02:13:42.428189 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:42.428199 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:42.428253 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:42.428451 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:42.428452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:42.428454 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:42.428517 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:42.428519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.429264 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.429265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:42.430255 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:42.430258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.430955 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.430956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:42.431157 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.431160 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:13:42.431316 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 02:13:42.431360 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":120,"threads_started":1}
I20260809 02:13:42.431660 24056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.428101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:57379 (local address 127.5.9.130:33837)
0809 02:13:42.428148 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:42.428150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:42.428261 (+ 111us) server_negotiation.cc:407] Connection header received
0809 02:13:42.428316 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:42.428317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:42.428328 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:42.428406 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:42.428582 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.429178 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.430355 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.430851 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.430924 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.430935 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:42.431511 (+ 576us) server_negotiation.cc:299] Negotiation successful
0809 02:13:42.431570 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:13:42.431609 23711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4d5a738fbc541d7afd1cad008be600c" candidate_uuid: "ba4ef8041a214ee78242297040c5d7e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb82ef3d2ac047e4968fb7599e581258" is_pre_election: true
I20260809 02:13:42.431751 23711 raft_consensus.cc:2454] T b4d5a738fbc541d7afd1cad008be600c P eb82ef3d2ac047e4968fb7599e581258 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba4ef8041a214ee78242297040c5d7e2 in term 0.
I20260809 02:13:42.431994 23922 leader_election.cc:302] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba4ef8041a214ee78242297040c5d7e2, eb82ef3d2ac047e4968fb7599e581258; no voters:
I20260809 02:13:42.432003 23851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4d5a738fbc541d7afd1cad008be600c" candidate_uuid: "ba4ef8041a214ee78242297040c5d7e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79ac667bbcc54234819d8f7f9a8d2117" is_pre_election: true
I20260809 02:13:42.432137 23851 raft_consensus.cc:2454] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba4ef8041a214ee78242297040c5d7e2 in term 0.
I20260809 02:13:42.432257 24065 raft_consensus.cc:2781] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:13:42.432343 24065 raft_consensus.cc:482] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:13:42.432374 24065 raft_consensus.cc:3037] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:42.433139 24065 raft_consensus.cc:504] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb82ef3d2ac047e4968fb7599e581258" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 32921 } } peers { permanent_uuid: "ba4ef8041a214ee78242297040c5d7e2" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35973 } } peers { permanent_uuid: "79ac667bbcc54234819d8f7f9a8d2117" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33837 } }
I20260809 02:13:42.433295 24065 leader_election.cc:288] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [CANDIDATE]: Term 1 election: Requested vote from peers eb82ef3d2ac047e4968fb7599e581258 (127.5.9.129:32921), 79ac667bbcc54234819d8f7f9a8d2117 (127.5.9.130:33837)
I20260809 02:13:42.433553 23851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4d5a738fbc541d7afd1cad008be600c" candidate_uuid: "ba4ef8041a214ee78242297040c5d7e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79ac667bbcc54234819d8f7f9a8d2117"
I20260809 02:13:42.433569 23711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4d5a738fbc541d7afd1cad008be600c" candidate_uuid: "ba4ef8041a214ee78242297040c5d7e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb82ef3d2ac047e4968fb7599e581258"
I20260809 02:13:42.433678 23711 raft_consensus.cc:3037] T b4d5a738fbc541d7afd1cad008be600c P eb82ef3d2ac047e4968fb7599e581258 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:42.433677 23851 raft_consensus.cc:3037] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:42.434489 23711 raft_consensus.cc:2454] T b4d5a738fbc541d7afd1cad008be600c P eb82ef3d2ac047e4968fb7599e581258 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba4ef8041a214ee78242297040c5d7e2 in term 1.
I20260809 02:13:42.434546 23851 raft_consensus.cc:2454] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba4ef8041a214ee78242297040c5d7e2 in term 1.
I20260809 02:13:42.434705 23922 leader_election.cc:302] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba4ef8041a214ee78242297040c5d7e2, eb82ef3d2ac047e4968fb7599e581258; no voters:
I20260809 02:13:42.434815 24065 raft_consensus.cc:2781] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:42.434998 24065 raft_consensus.cc:686] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [term 1 LEADER]: Becoming Leader. State: Replica: ba4ef8041a214ee78242297040c5d7e2, State: Running, Role: LEADER
I20260809 02:13:42.435103 24065 consensus_queue.cc:237] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb82ef3d2ac047e4968fb7599e581258" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 32921 } } peers { permanent_uuid: "ba4ef8041a214ee78242297040c5d7e2" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35973 } } peers { permanent_uuid: "79ac667bbcc54234819d8f7f9a8d2117" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33837 } }
I20260809 02:13:42.436036 23562 catalog_manager.cc:5495] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba4ef8041a214ee78242297040c5d7e2 (127.5.9.131). New cstate: current_term: 1 leader_uuid: "ba4ef8041a214ee78242297040c5d7e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb82ef3d2ac047e4968fb7599e581258" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 32921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba4ef8041a214ee78242297040c5d7e2" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79ac667bbcc54234819d8f7f9a8d2117" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33837 } health_report { overall_health: UNKNOWN } } }
I20260809 02:13:42.448493 24053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.446042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48898 (local address 127.5.9.131:35973)
0809 02:13:42.446110 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:42.446112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:42.446121 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:13:42.446157 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:42.446158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:42.446176 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:42.446240 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:13:42.446360 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.446847 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.447589 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:42.447823 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:42.447878 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:42.448104 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:13:42.448262 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:13:42.448325 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:42.448346 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 02:13:42.448362 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:13:42.450572 23851 raft_consensus.cc:1261] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117 [term 1 FOLLOWER]: Refusing update from remote peer ba4ef8041a214ee78242297040c5d7e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:13:42.450932 23711 raft_consensus.cc:1261] T b4d5a738fbc541d7afd1cad008be600c P eb82ef3d2ac047e4968fb7599e581258 [term 1 FOLLOWER]: Refusing update from remote peer ba4ef8041a214ee78242297040c5d7e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 02:13:42.450979 23902 tablet.cc:2369] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:13:42.450979 24065 consensus_queue.cc:1035] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79ac667bbcc54234819d8f7f9a8d2117" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 33837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:13:42.451279 24065 consensus_queue.cc:1035] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb82ef3d2ac047e4968fb7599e581258" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 32921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:13:42.453954 24074 mvcc.cc:205] Tried to move back new op lower bound from 7316445685554520064 to 7316445685494951936. Current Snapshot: MvccSnapshot[applied={T|T < 7316445685554520064}]
I20260809 02:13:42.454044 24072 mvcc.cc:205] Tried to move back new op lower bound from 7316445685554520064 to 7316445685494951936. Current Snapshot: MvccSnapshot[applied={T|T < 7316445685554520064}]
I20260809 02:13:42.456166 23560 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:13:42.456290 23560 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:13:42.457054 23560 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e0bbfd0c2ad49edae5a5b6a6e4b4038: Sending DeleteTablet for 3 replicas of tablet b4d5a738fbc541d7afd1cad008be600c
I20260809 02:13:42.457473 23973 tablet_service.cc:1514] Processing DeleteTablet for tablet b4d5a738fbc541d7afd1cad008be600c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:42 UTC) from {username='slave'} at 127.0.0.1:48888
I20260809 02:13:42.457480 23829 tablet_service.cc:1514] Processing DeleteTablet for tablet b4d5a738fbc541d7afd1cad008be600c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:42 UTC) from {username='slave'} at 127.0.0.1:58218
I20260809 02:13:42.457705 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 23624
I20260809 02:13:42.457759 23691 tablet_service.cc:1514] Processing DeleteTablet for tablet b4d5a738fbc541d7afd1cad008be600c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:42 UTC) from {username='slave'} at 127.0.0.1:57048
I20260809 02:13:42.459355 24082 tablet_replica.cc:330] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2: stopping tablet replica
I20260809 02:13:42.459488 24082 raft_consensus.cc:2229] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:13:42.459661 24082 raft_consensus.cc:2258] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:13:42.462328 24082 ts_tablet_manager.cc:1905] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:13:42.463912 24082 ts_tablet_manager.cc:1918] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:13:42.463980 24082 log.cc:1196] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/b4d5a738fbc541d7afd1cad008be600c
I20260809 02:13:42.464239 24082 ts_tablet_manager.cc:1939] T b4d5a738fbc541d7afd1cad008be600c P ba4ef8041a214ee78242297040c5d7e2: Deleting consensus metadata
I20260809 02:13:42.464649 23545 catalog_manager.cc:4841] TS ba4ef8041a214ee78242297040c5d7e2 (127.5.9.131:35973): tablet b4d5a738fbc541d7afd1cad008be600c (table test-table [id=920a4bb2ec5f4690983990ac7d223245]) successfully deleted
I20260809 02:13:42.465762 24081 tablet_replica.cc:330] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117: stopping tablet replica
I20260809 02:13:42.465910 24081 raft_consensus.cc:2229] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:13:42.466086 24081 raft_consensus.cc:2258] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:13:42.466549 24081 ts_tablet_manager.cc:1905] T b4d5a738fbc541d7afd1cad008be600c P 79ac667bbcc54234819d8f7f9a8d2117: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 02:13:42.466854 23547 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:32921 (error 108)
I20260809 02:13:42.467373 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.467216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:32921 (local address 127.0.0.1:57058)
0809 02:13:42.467282 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:42.467336 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:32921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 02:13:42.467505 23547 catalog_manager.cc:4584] TS eb82ef3d2ac047e4968fb7599e581258 (127.5.9.129:32921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b4d5a738fbc541d7afd1cad008be600c: Network error: Client connection negotiation failed: client connection to 127.5.9.129:32921: connect: Connection refused (error 111)
I20260809 02:13:42.467850 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 23762
I20260809 02:13:42.475185 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 23904
I20260809 02:13:42.475310 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:42.475152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:33837 (local address 127.0.0.1:58234)
0809 02:13:42.475219 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:42.475264 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.130:33837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 02:13:42.481495 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 23529
2026-08-09T02:13:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (769 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:13:42 dist-test-slave-p3t5 krb5kdc[24086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:13:42 dist-test-slave-p3t5 krb5kdc[24086](info): set up 2 sockets
Aug 09 02:13:42 dist-test-slave-p3t5 krb5kdc[24086](info): commencing operation
krb5kdc: starting...
W20260809 02:13:43.557950 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:13:43 dist-test-slave-p3t5 krb5kdc[24086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:13:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:13:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:13:43.689785 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:41087
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:41087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:43.775125 24100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:43.775404 24100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:43.775444 24100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:43.776608 24100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:13:43.776685 24100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:43.776713 24100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:43.776746 24100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:13:43.776772 24100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:13:43.778388 24100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:41087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:41087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:43.778702 24100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:43.778960 24100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:43.784332 24107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:43.784508 24100 server_base.cc:1034] running on GCE node
W20260809 02:13:43.784329 24108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:43.784348 24111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:43.784961 24100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:43.785287 24100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:43.786474 24100 hybrid_clock.cc:648] HybridClock initialized: now 1786241623786450 us; error 39 us; skew 500 ppm
Aug 09 02:13:43 dist-test-slave-p3t5 krb5kdc[24086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241623, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:43.789433 24100 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:13:43.789976 24100 webserver.cc:460] Webserver started at http://127.5.9.190:39443/ using document root <none> and password file <none>
I20260809 02:13:43.790247 24100 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:43.790298 24100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:43.790443 24100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:43.791612 24100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "b06bc5abef424ed8b7f6b66c863c7e19"
format_stamp: "Formatted at 2026-08-09 02:13:43 on dist-test-slave-p3t5"
server_key: "71e29ecb594af1fbd1349c5e2125f75c"
server_key_iv: "20d13944423ce25f547cae83aaa8ceb2"
server_key_version: "encryptionkey@0"
I20260809 02:13:43.792021 24100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "b06bc5abef424ed8b7f6b66c863c7e19"
format_stamp: "Formatted at 2026-08-09 02:13:43 on dist-test-slave-p3t5"
server_key: "71e29ecb594af1fbd1349c5e2125f75c"
server_key_iv: "20d13944423ce25f547cae83aaa8ceb2"
server_key_version: "encryptionkey@0"
I20260809 02:13:43.793562 24100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:13:43.794457 24119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:43.794826 24100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:13:43.794941 24100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "b06bc5abef424ed8b7f6b66c863c7e19"
format_stamp: "Formatted at 2026-08-09 02:13:43 on dist-test-slave-p3t5"
server_key: "71e29ecb594af1fbd1349c5e2125f75c"
server_key_iv: "20d13944423ce25f547cae83aaa8ceb2"
server_key_version: "encryptionkey@0"
I20260809 02:13:43.795023 24100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:43.811888 24100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:43.814229 24100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:43.814388 24100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:43.818794 24100 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:41087
I20260809 02:13:43.818874 24174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:41087 every 8 connection(s)
I20260809 02:13:43.819468 24100 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:13:43.820187 24175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:43.822510 24175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19: Bootstrap starting.
I20260809 02:13:43.823078 24175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:43.823319 24175 log.cc:824] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:43.824067 24175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19: No bootstrap required, opened a new log
I20260809 02:13:43.825687 24175 raft_consensus.cc:348] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06bc5abef424ed8b7f6b66c863c7e19" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41087 } }
I20260809 02:13:43.825850 24175 raft_consensus.cc:374] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:43.825879 24175 raft_consensus.cc:729] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b06bc5abef424ed8b7f6b66c863c7e19, State: Initialized, Role: FOLLOWER
I20260809 02:13:43.826040 24175 consensus_queue.cc:260] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06bc5abef424ed8b7f6b66c863c7e19" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41087 } }
I20260809 02:13:43.826114 24175 raft_consensus.cc:388] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:13:43.826083 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 24100
I20260809 02:13:43.826159 24175 raft_consensus.cc:482] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:13:43.826210 24175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:43.826205 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:13:43.826428 5158 external_mini_cluster.cc:1428] Setting key 5bc8b4e17360dbd1fb1eb6740b0fdd76
I20260809 02:13:43.827445 24175 raft_consensus.cc:504] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06bc5abef424ed8b7f6b66c863c7e19" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41087 } }
I20260809 02:13:43.827580 24175 leader_election.cc:302] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b06bc5abef424ed8b7f6b66c863c7e19; no voters:
I20260809 02:13:43.827749 24175 leader_election.cc:288] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:13:43.827827 24181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:43.828018 24181 raft_consensus.cc:686] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [term 1 LEADER]: Becoming Leader. State: Replica: b06bc5abef424ed8b7f6b66c863c7e19, State: Running, Role: LEADER
I20260809 02:13:43.828022 24175 sys_catalog.cc:564] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:13:43.829519 24181 consensus_queue.cc:237] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06bc5abef424ed8b7f6b66c863c7e19" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41087 } }
I20260809 02:13:43.830253 24183 sys_catalog.cc:455] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b06bc5abef424ed8b7f6b66c863c7e19. Latest consensus state: current_term: 1 leader_uuid: "b06bc5abef424ed8b7f6b66c863c7e19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06bc5abef424ed8b7f6b66c863c7e19" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41087 } } }
I20260809 02:13:43.830538 24183 sys_catalog.cc:458] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:43.830323 24182 sys_catalog.cc:455] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b06bc5abef424ed8b7f6b66c863c7e19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06bc5abef424ed8b7f6b66c863c7e19" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41087 } } }
I20260809 02:13:43.830749 24182 sys_catalog.cc:458] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19 [sys.catalog]: This master's current role is: LEADER
W20260809 02:13:43.830965 24196 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:13:43.831037 24196 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:13:43.831133 24198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:13:43.831350 24198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:13:43.833127 24198 catalog_manager.cc:1347] Generated new cluster ID: c39604582f4a4cbab743decdb38419fd
I20260809 02:13:43.833180 24198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:13:43 dist-test-slave-p3t5 krb5kdc[24086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:13:43.838701 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:43.827359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46148 (local address 127.5.9.190:41087)
0809 02:13:43.827605 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:43.827615 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:43.827650 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:13:43.827788 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:43.827795 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:43.827861 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:43.828793 (+ 932us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:43.828902 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:43.830171 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:43.831171 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:43.831758 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:43.831883 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:43.834900 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:43.834904 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:43.834905 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:43.834918 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:43.836900 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:43.837348 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:43.837353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:43.837354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:43.837471 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:43.837754 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:43.837755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:43.837755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:43.838045 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:43.838258 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:43.838347 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 02:13:43.838510 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":122,"threads_started":1}
I20260809 02:13:43.869781 24198 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:13:43.870402 24198 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:13:43.880050 24198 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19: Generated new TSK 0
I20260809 02:13:43.880292 24198 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:13:43.924686 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41087
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:44.033983 24204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:44.034209 24204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:44.034235 24204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:44.035468 24204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:44.035516 24204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:44.035543 24204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:13:44.037137 24204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:44.037423 24204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:44.037721 24204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:44.043016 24211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:44.043058 24210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:44.043072 24213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:44.043262 24204 server_base.cc:1034] running on GCE node
I20260809 02:13:44.043491 24204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:44.043766 24204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:44.044943 24204 hybrid_clock.cc:648] HybridClock initialized: now 1786241624044916 us; error 44 us; skew 500 ppm
Aug 09 02:13:44 dist-test-slave-p3t5 krb5kdc[24086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:44.048625 24204 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:13:44.049192 24204 webserver.cc:460] Webserver started at http://127.5.9.129:42979/ using document root <none> and password file <none>
I20260809 02:13:44.049476 24204 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:44.049551 24204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:44.049723 24204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:44.050884 24204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "c4efef6fd7704b21acf3b43646e8d99b"
format_stamp: "Formatted at 2026-08-09 02:13:44 on dist-test-slave-p3t5"
server_key: "49e94dcd313b3b4272f877e29a32aa9c"
server_key_iv: "b420ec60e99b0807fd57e7cac893ea2d"
server_key_version: "encryptionkey@0"
I20260809 02:13:44.051286 24204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "c4efef6fd7704b21acf3b43646e8d99b"
format_stamp: "Formatted at 2026-08-09 02:13:44 on dist-test-slave-p3t5"
server_key: "49e94dcd313b3b4272f877e29a32aa9c"
server_key_iv: "b420ec60e99b0807fd57e7cac893ea2d"
server_key_version: "encryptionkey@0"
I20260809 02:13:44.052798 24204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:13:44.053774 24222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:44.053985 24204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:44.054090 24204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "c4efef6fd7704b21acf3b43646e8d99b"
format_stamp: "Formatted at 2026-08-09 02:13:44 on dist-test-slave-p3t5"
server_key: "49e94dcd313b3b4272f877e29a32aa9c"
server_key_iv: "b420ec60e99b0807fd57e7cac893ea2d"
server_key_version: "encryptionkey@0"
I20260809 02:13:44.054181 24204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:44.065465 24204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:44.067503 24204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:44.067683 24204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:44.067952 24204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:44.068282 24204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:44.068341 24204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:44.068377 24204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:44.068423 24204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:44.074714 24204 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:37511
I20260809 02:13:44.074771 24342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:37511 every 8 connection(s)
I20260809 02:13:44.075099 24204 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:13:44 dist-test-slave-p3t5 krb5kdc[24086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:13:44.082720 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 24204
I20260809 02:13:44.082856 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:13:44.083087 5158 external_mini_cluster.cc:1428] Setting key 63c367e71b11116858d25dc8b01880b6
I20260809 02:13:44.085657 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.075913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:49651 (local address 127.5.9.190:41087)
0809 02:13:44.076004 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:44.076006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:44.076165 (+ 159us) server_negotiation.cc:407] Connection header received
0809 02:13:44.076718 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:44.076719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:44.076733 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:44.076830 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:44.077871 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.078388 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.079473 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.079683 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.079762 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.082176 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:44.082178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:44.082178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:44.082186 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:44.084090 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:44.084606 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:44.084607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:44.084608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:44.084665 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:44.084966 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:44.084967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:44.084967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:44.085073 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:44.085124 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:44.085460 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 02:13:44.085543 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:13:44.086020 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.075811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:41087 (local address 127.5.9.129:49651)
0809 02:13:44.076022 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:44.076071 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:44.076605 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:44.076890 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:44.076894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:44.077503 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:44.077743 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:44.077748 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.078492 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.078494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:44.079372 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:44.079375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.079785 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.079786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:44.079920 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.080477 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:13:44.080486 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:44.082046 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:44.084173 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:44.084181 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:44.084184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:44.084526 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:44.084728 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:44.084729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:44.084730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:44.084895 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:44.085138 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:44.085144 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:44.085320 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 02:13:44.085438 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 02:13:44.085656 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":89,"threads_started":1}
I20260809 02:13:44.086572 24343 heartbeater.cc:347] Connected to a master server at 127.5.9.190:41087
I20260809 02:13:44.086697 24343 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:44.086915 24343 heartbeater.cc:511] Master 127.5.9.190:41087 requested a full tablet report, sending...
I20260809 02:13:44.087512 24138 ts_manager.cc:194] Registered new tserver with Master: c4efef6fd7704b21acf3b43646e8d99b (127.5.9.129:37511)
I20260809 02:13:44.088145 24138 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:49651
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:13:44.116680 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41087
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:44.204257 24348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:44.204493 24348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:44.204519 24348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:44.205812 24348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:44.205865 24348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:44.205894 24348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:13:44.207473 24348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:44.207754 24348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:44.208046 24348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:44.213521 24357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:44.213567 24353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:44.213711 24348 server_base.cc:1034] running on GCE node
W20260809 02:13:44.213544 24354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:44.214012 24348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:44.214295 24348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:44.215441 24348 hybrid_clock.cc:648] HybridClock initialized: now 1786241624215427 us; error 26 us; skew 500 ppm
Aug 09 02:13:44 dist-test-slave-p3t5 krb5kdc[24086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:44.218287 24348 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:13:44.218796 24348 webserver.cc:460] Webserver started at http://127.5.9.130:44019/ using document root <none> and password file <none>
I20260809 02:13:44.219035 24348 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:44.219095 24348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:44.219235 24348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:44.220343 24348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "3f3d3c272db046b2adf30f3435b5ee8a"
format_stamp: "Formatted at 2026-08-09 02:13:44 on dist-test-slave-p3t5"
server_key: "b4e841d4673d8635d8e7aaadb95bd2be"
server_key_iv: "8e64bae7db7c9b5f8a33dd27331c9282"
server_key_version: "encryptionkey@0"
I20260809 02:13:44.220727 24348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "3f3d3c272db046b2adf30f3435b5ee8a"
format_stamp: "Formatted at 2026-08-09 02:13:44 on dist-test-slave-p3t5"
server_key: "b4e841d4673d8635d8e7aaadb95bd2be"
server_key_iv: "8e64bae7db7c9b5f8a33dd27331c9282"
server_key_version: "encryptionkey@0"
I20260809 02:13:44.222301 24348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:13:44.223136 24364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:44.223354 24348 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:44.223450 24348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "3f3d3c272db046b2adf30f3435b5ee8a"
format_stamp: "Formatted at 2026-08-09 02:13:44 on dist-test-slave-p3t5"
server_key: "b4e841d4673d8635d8e7aaadb95bd2be"
server_key_iv: "8e64bae7db7c9b5f8a33dd27331c9282"
server_key_version: "encryptionkey@0"
I20260809 02:13:44.223529 24348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:44.238327 24348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:44.240518 24348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:44.240693 24348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:44.240970 24348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:44.241299 24348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:44.241336 24348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:44.241382 24348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:44.241410 24348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:44.247591 24348 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:37165
I20260809 02:13:44.247648 24485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:37165 every 8 connection(s)
I20260809 02:13:44.247974 24348 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:13:44.253281 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 24348
I20260809 02:13:44.253429 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:13:44.253659 5158 external_mini_cluster.cc:1428] Setting key 9ec26bfe4d17ac1ff2cd80879371f894
Aug 09 02:13:44 dist-test-slave-p3t5 krb5kdc[24086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:13:44.258188 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.248749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:58919 (local address 127.5.9.190:41087)
0809 02:13:44.248811 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:44.248813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:44.249010 (+ 197us) server_negotiation.cc:407] Connection header received
0809 02:13:44.249510 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:44.249511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:44.249521 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:44.249580 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:44.250630 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.251141 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.252174 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.252458 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.252515 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.255094 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:44.255096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:44.255096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:44.255104 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:44.256473 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:44.256916 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:44.256917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:44.256918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:44.257012 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:44.257336 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:44.257336 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:44.257337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:44.257487 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:44.257562 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:44.257921 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 02:13:44.258036 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 02:13:44.258324 24488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.248717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:41087 (local address 127.5.9.130:58919)
0809 02:13:44.248879 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:44.248922 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:44.249404 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:44.249690 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:44.249694 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:44.250327 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:44.250526 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:44.250531 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.251228 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.251229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:44.252032 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:44.252034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.252544 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.252545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:44.252656 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.253235 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:13:44.253243 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:44.254984 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:44.256577 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:44.256584 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:44.256587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:44.256826 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:44.257090 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:44.257091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:44.257091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:44.257259 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:44.257582 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:44.257589 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:44.257771 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 02:13:44.257868 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 02:13:44.258005 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260809 02:13:44.259006 24486 heartbeater.cc:347] Connected to a master server at 127.5.9.190:41087
I20260809 02:13:44.259124 24486 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:44.259315 24486 heartbeater.cc:511] Master 127.5.9.190:41087 requested a full tablet report, sending...
I20260809 02:13:44.259749 24138 ts_manager.cc:194] Registered new tserver with Master: 3f3d3c272db046b2adf30f3435b5ee8a (127.5.9.130:37165)
I20260809 02:13:44.260155 24138 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:58919
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:13:44.285399 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41087
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:44.372256 24491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:44.372494 24491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:44.372519 24491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:44.373679 24491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:44.373720 24491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:44.373749 24491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:13:44.375221 24491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:41087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:44.375447 24491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:44.375638 24491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:44.380867 24499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:44.380882 24497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:44.380977 24496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:44.381103 24491 server_base.cc:1034] running on GCE node
I20260809 02:13:44.381305 24491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:44.381573 24491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:44.382769 24491 hybrid_clock.cc:648] HybridClock initialized: now 1786241624382746 us; error 48 us; skew 500 ppm
Aug 09 02:13:44 dist-test-slave-p3t5 krb5kdc[24086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:44.385609 24491 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:13:44.386124 24491 webserver.cc:460] Webserver started at http://127.5.9.131:39265/ using document root <none> and password file <none>
I20260809 02:13:44.386384 24491 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:44.386430 24491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:44.386579 24491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:44.387701 24491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "d5752a81d89045af8a05ff1fabd2a401"
format_stamp: "Formatted at 2026-08-09 02:13:44 on dist-test-slave-p3t5"
server_key: "0efa81c0127a2e883558c03f0a327948"
server_key_iv: "ecb4612e009e5606fd808d72972bdc85"
server_key_version: "encryptionkey@0"
I20260809 02:13:44.388098 24491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "d5752a81d89045af8a05ff1fabd2a401"
format_stamp: "Formatted at 2026-08-09 02:13:44 on dist-test-slave-p3t5"
server_key: "0efa81c0127a2e883558c03f0a327948"
server_key_iv: "ecb4612e009e5606fd808d72972bdc85"
server_key_version: "encryptionkey@0"
I20260809 02:13:44.389681 24491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:13:44.390588 24506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:44.390790 24491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:13:44.390930 24491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "d5752a81d89045af8a05ff1fabd2a401"
format_stamp: "Formatted at 2026-08-09 02:13:44 on dist-test-slave-p3t5"
server_key: "0efa81c0127a2e883558c03f0a327948"
server_key_iv: "ecb4612e009e5606fd808d72972bdc85"
server_key_version: "encryptionkey@0"
I20260809 02:13:44.391017 24491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:44.410076 24491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:44.412611 24491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:44.412774 24491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:44.413041 24491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:44.413381 24491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:44.413439 24491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:44.413486 24491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:44.413513 24491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:44.419747 24491 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:37217
I20260809 02:13:44.419797 24628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:37217 every 8 connection(s)
I20260809 02:13:44.420187 24491 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:13:44.421696 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 24491
I20260809 02:13:44.421795 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:13:44.422001 5158 external_mini_cluster.cc:1428] Setting key 24d0abea385004a21f72ea1520185362
Aug 09 02:13:44 dist-test-slave-p3t5 krb5kdc[24086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:13:44.430701 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.421001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:36715 (local address 127.5.9.190:41087)
0809 02:13:44.421079 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:44.421081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:44.421305 (+ 224us) server_negotiation.cc:407] Connection header received
0809 02:13:44.421991 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:44.421993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:44.422006 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:44.422064 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:44.423290 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.423807 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.424833 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.425051 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.425132 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.427614 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:44.427615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:44.427616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:44.427623 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:44.429231 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:44.429728 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:44.429730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:44.429730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:44.429787 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:44.430028 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:44.430028 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:44.430028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:44.430144 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:44.430193 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:44.430554 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 02:13:44.430632 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:13:44.430744 24631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.420973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:41087 (local address 127.5.9.131:36715)
0809 02:13:44.421172 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:44.421215 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:44.421867 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:44.422138 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:44.422143 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:44.422902 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:44.423187 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:44.423193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.423909 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.423917 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:44.424733 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:44.424735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.425138 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.425139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:44.425289 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.425926 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:13:44.425935 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:44.427514 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:44.429303 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:44.429309 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:44.429311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:44.429548 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:44.429850 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:44.429850 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:44.429851 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:44.429962 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:44.430199 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:44.430204 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:44.430333 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 02:13:44.430423 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 02:13:44.430561 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
I20260809 02:13:44.431301 24629 heartbeater.cc:347] Connected to a master server at 127.5.9.190:41087
I20260809 02:13:44.431424 24629 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:44.431643 24629 heartbeater.cc:511] Master 127.5.9.190:41087 requested a full tablet report, sending...
I20260809 02:13:44.432076 24138 ts_manager.cc:194] Registered new tserver with Master: d5752a81d89045af8a05ff1fabd2a401 (127.5.9.131:37217)
I20260809 02:13:44.432489 24138 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:36715
I20260809 02:13:44.434093 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:13:44.442452 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.435545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46150 (local address 127.5.9.190:41087)
0809 02:13:44.435621 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:44.435624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:44.435729 (+ 105us) server_negotiation.cc:407] Connection header received
0809 02:13:44.435814 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:44.435816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:44.435825 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:44.435882 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:44.436491 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.436991 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.437801 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.438017 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.438096 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.439591 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:44.439592 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:44.439593 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:44.439601 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:44.441075 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:44.441510 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:44.441511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:44.441512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:44.441579 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:44.441839 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:44.441840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:44.441841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:44.442013 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:44.442154 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:44.442271 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 02:13:44.442340 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:13:44.444342 24138 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:13:44.446074 24138 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:13:44.452808 24641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.448706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:37217 (local address 127.0.0.1:56802)
0809 02:13:44.448893 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:44.448935 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:44.449002 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:44.449389 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:44.449392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:44.449395 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:44.449482 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:44.449484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.450540 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.450541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:44.451673 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:44.451676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.452448 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.452449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:44.452545 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.452559 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 02:13:44.452639 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 02:13:44.452720 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"spinlock_wait_cycles":768,"thread_start_us":89,"threads_started":1}
I20260809 02:13:44.452808 24642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.448800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56802 (local address 127.5.9.131:37217)
0809 02:13:44.449180 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:44.449187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:44.449202 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:13:44.449236 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:44.449237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:44.449249 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:44.449371 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:44.449575 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.450436 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.451749 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.452357 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.452440 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.452498 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:44.452670 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 02:13:44.452715 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":101,"threads_started":1}
I20260809 02:13:44.453272 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.448706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:37511 (local address 127.0.0.1:42608)
0809 02:13:44.448994 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:44.449007 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:44.449069 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:44.449732 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:44.449733 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:44.449735 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:44.449773 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:44.449775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.451025 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.451026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:44.451999 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:44.452001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.453015 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.453016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:44.453088 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.453092 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:13:44.453151 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 02:13:44.453192 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":3,"thread_start_us":74,"threads_started":1}
I20260809 02:13:44.453421 24644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.448947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42608 (local address 127.5.9.129:37511)
0809 02:13:44.449523 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:44.449527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:44.449537 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:13:44.449575 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:44.449577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:44.449592 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:44.449731 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:44.449842 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.450925 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.452082 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.452907 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.453006 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.453063 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:44.453164 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 02:13:44.453199 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":95,"threads_started":1}
I20260809 02:13:44.453933 24556 tablet_service.cc:1467] Processing CreateTablet for tablet 9c4dcacdf11644e4a8da2739866d2595 (DEFAULT_TABLE table=test-table [id=0c74e3e300924c40a1cbde22dfbaad02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:44.454107 24273 tablet_service.cc:1467] Processing CreateTablet for tablet 9c4dcacdf11644e4a8da2739866d2595 (DEFAULT_TABLE table=test-table [id=0c74e3e300924c40a1cbde22dfbaad02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:44.454331 24556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c4dcacdf11644e4a8da2739866d2595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:44.454362 24273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c4dcacdf11644e4a8da2739866d2595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:44.454363 24643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.448928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:37165 (local address 127.0.0.1:34984)
0809 02:13:44.449148 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:44.449157 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:44.449203 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:44.449600 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:44.449600 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:44.449603 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:44.449698 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:44.449701 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.452068 (+ 2367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.452069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:44.453103 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:44.453106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.454119 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.454120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:44.454182 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.454187 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:13:44.454237 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 02:13:44.454285 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":1,"thread_start_us":70,"threads_started":1}
I20260809 02:13:44.454596 24645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.449151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34984 (local address 127.5.9.130:37165)
0809 02:13:44.449358 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:44.449362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:44.449372 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:13:44.449421 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:44.449423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:44.449437 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:44.449513 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:44.449791 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.450909 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.453197 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.453957 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.454027 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.454080 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:44.454463 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 02:13:44.454512 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":103,"threads_started":1}
I20260809 02:13:44.457103 24647 tablet_bootstrap.cc:492] T 9c4dcacdf11644e4a8da2739866d2595 P c4efef6fd7704b21acf3b43646e8d99b: Bootstrap starting.
I20260809 02:13:44.457499 24414 tablet_service.cc:1467] Processing CreateTablet for tablet 9c4dcacdf11644e4a8da2739866d2595 (DEFAULT_TABLE table=test-table [id=0c74e3e300924c40a1cbde22dfbaad02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:44.457876 24414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c4dcacdf11644e4a8da2739866d2595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:44.457947 24647 tablet_bootstrap.cc:654] T 9c4dcacdf11644e4a8da2739866d2595 P c4efef6fd7704b21acf3b43646e8d99b: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:44.458460 24647 log.cc:824] T 9c4dcacdf11644e4a8da2739866d2595 P c4efef6fd7704b21acf3b43646e8d99b: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:44.458846 24646 tablet_bootstrap.cc:492] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401: Bootstrap starting.
I20260809 02:13:44.459407 24646 tablet_bootstrap.cc:654] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:44.459877 24646 log.cc:824] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:44.460415 24647 tablet_bootstrap.cc:492] T 9c4dcacdf11644e4a8da2739866d2595 P c4efef6fd7704b21acf3b43646e8d99b: No bootstrap required, opened a new log
I20260809 02:13:44.460502 24647 ts_tablet_manager.cc:1397] T 9c4dcacdf11644e4a8da2739866d2595 P c4efef6fd7704b21acf3b43646e8d99b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:13:44.460677 24646 tablet_bootstrap.cc:492] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401: No bootstrap required, opened a new log
I20260809 02:13:44.460757 24646 ts_tablet_manager.cc:1397] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:13:44.460712 24650 tablet_bootstrap.cc:492] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a: Bootstrap starting.
I20260809 02:13:44.461508 24650 tablet_bootstrap.cc:654] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:44.462158 24650 log.cc:824] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:44.462244 24646 raft_consensus.cc:348] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5752a81d89045af8a05ff1fabd2a401" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37217 } } peers { permanent_uuid: "c4efef6fd7704b21acf3b43646e8d99b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 37511 } } peers { permanent_uuid: "3f3d3c272db046b2adf30f3435b5ee8a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37165 } }
I20260809 02:13:44.462425 24646 raft_consensus.cc:374] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:44.462476 24646 raft_consensus.cc:729] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5752a81d89045af8a05ff1fabd2a401, State: Initialized, Role: FOLLOWER
I20260809 02:13:44.462585 24646 consensus_queue.cc:260] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5752a81d89045af8a05ff1fabd2a401" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37217 } } peers { permanent_uuid: "c4efef6fd7704b21acf3b43646e8d99b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 37511 } } peers { permanent_uuid: "3f3d3c272db046b2adf30f3435b5ee8a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37165 } }
I20260809 02:13:44.462893 24646 ts_tablet_manager.cc:1428] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:13:44.462905 24629 heartbeater.cc:503] Master 127.5.9.190:41087 was elected leader, sending a full tablet report...
I20260809 02:13:44.462809 24647 raft_consensus.cc:348] T 9c4dcacdf11644e4a8da2739866d2595 P c4efef6fd7704b21acf3b43646e8d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5752a81d89045af8a05ff1fabd2a401" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37217 } } peers { permanent_uuid: "c4efef6fd7704b21acf3b43646e8d99b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 37511 } } peers { permanent_uuid: "3f3d3c272db046b2adf30f3435b5ee8a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37165 } }
I20260809 02:13:44.462997 24647 raft_consensus.cc:374] T 9c4dcacdf11644e4a8da2739866d2595 P c4efef6fd7704b21acf3b43646e8d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:44.463011 24650 tablet_bootstrap.cc:492] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a: No bootstrap required, opened a new log
I20260809 02:13:44.463038 24647 raft_consensus.cc:729] T 9c4dcacdf11644e4a8da2739866d2595 P c4efef6fd7704b21acf3b43646e8d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4efef6fd7704b21acf3b43646e8d99b, State: Initialized, Role: FOLLOWER
I20260809 02:13:44.463068 24650 ts_tablet_manager.cc:1397] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:13:44.463147 24647 consensus_queue.cc:260] T 9c4dcacdf11644e4a8da2739866d2595 P c4efef6fd7704b21acf3b43646e8d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5752a81d89045af8a05ff1fabd2a401" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37217 } } peers { permanent_uuid: "c4efef6fd7704b21acf3b43646e8d99b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 37511 } } peers { permanent_uuid: "3f3d3c272db046b2adf30f3435b5ee8a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37165 } }
I20260809 02:13:44.463428 24647 ts_tablet_manager.cc:1428] T 9c4dcacdf11644e4a8da2739866d2595 P c4efef6fd7704b21acf3b43646e8d99b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:13:44.463477 24343 heartbeater.cc:503] Master 127.5.9.190:41087 was elected leader, sending a full tablet report...
I20260809 02:13:44.464958 24650 raft_consensus.cc:348] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5752a81d89045af8a05ff1fabd2a401" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37217 } } peers { permanent_uuid: "c4efef6fd7704b21acf3b43646e8d99b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 37511 } } peers { permanent_uuid: "3f3d3c272db046b2adf30f3435b5ee8a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37165 } }
I20260809 02:13:44.465097 24650 raft_consensus.cc:374] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:44.465118 24650 raft_consensus.cc:729] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f3d3c272db046b2adf30f3435b5ee8a, State: Initialized, Role: FOLLOWER
I20260809 02:13:44.465231 24650 consensus_queue.cc:260] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5752a81d89045af8a05ff1fabd2a401" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37217 } } peers { permanent_uuid: "c4efef6fd7704b21acf3b43646e8d99b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 37511 } } peers { permanent_uuid: "3f3d3c272db046b2adf30f3435b5ee8a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37165 } }
I20260809 02:13:44.465469 24650 ts_tablet_manager.cc:1428] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 02:13:44.465538 24486 heartbeater.cc:503] Master 127.5.9.190:41087 was elected leader, sending a full tablet report...
W20260809 02:13:44.498546 24487 tablet.cc:2369] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:13:44.576000 24344 tablet.cc:2369] T 9c4dcacdf11644e4a8da2739866d2595 P c4efef6fd7704b21acf3b43646e8d99b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:13:44.670841 24630 tablet.cc:2369] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:13:44.825814 24652 raft_consensus.cc:482] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:13:44.825961 24652 raft_consensus.cc:504] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5752a81d89045af8a05ff1fabd2a401" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37217 } } peers { permanent_uuid: "c4efef6fd7704b21acf3b43646e8d99b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 37511 } } peers { permanent_uuid: "3f3d3c272db046b2adf30f3435b5ee8a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37165 } }
I20260809 02:13:44.826406 24652 leader_election.cc:288] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4efef6fd7704b21acf3b43646e8d99b (127.5.9.129:37511), 3f3d3c272db046b2adf30f3435b5ee8a (127.5.9.130:37165)
I20260809 02:13:44.830001 24631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.826668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:37511 (local address 127.5.9.131:56055)
0809 02:13:44.826751 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:44.826766 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:44.826840 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:44.827080 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:44.827081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:44.827083 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:44.827177 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:44.827180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.827936 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.827937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:44.828902 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:44.828905 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.829714 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.829716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:44.829805 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.829808 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:13:44.829862 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 02:13:44.829897 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 02:13:44.830063 24644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.826742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:56055 (local address 127.5.9.129:37511)
0809 02:13:44.826804 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:44.826806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:44.826821 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:13:44.826952 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:44.826953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:44.826971 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:44.827057 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:44.827259 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.827851 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.829047 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.829648 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.829732 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.829746 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:44.829899 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 02:13:44.829952 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 02:13:44.830107 24645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.826888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:41003 (local address 127.5.9.130:37165)
0809 02:13:44.826942 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:44.826944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:44.827092 (+ 148us) server_negotiation.cc:407] Connection header received
0809 02:13:44.827146 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:44.827147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:44.827160 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:44.827227 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:44.827443 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.827951 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.829197 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.829670 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.829744 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.829756 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:44.829986 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 02:13:44.830034 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 02:13:44.830164 24655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.826825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:37165 (local address 127.5.9.131:41003)
0809 02:13:44.827013 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:44.827025 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:44.827086 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:44.827259 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:44.827260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:44.827262 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:44.827336 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:44.827338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.828078 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.828079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:44.829116 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:44.829119 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.829769 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.829771 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:44.829876 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.829883 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 02:13:44.829948 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 02:13:44.829990 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":112,"threads_started":1}
I20260809 02:13:44.830421 24294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c4dcacdf11644e4a8da2739866d2595" candidate_uuid: "d5752a81d89045af8a05ff1fabd2a401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4efef6fd7704b21acf3b43646e8d99b" is_pre_election: true
I20260809 02:13:44.830624 24294 raft_consensus.cc:2454] T 9c4dcacdf11644e4a8da2739866d2595 P c4efef6fd7704b21acf3b43646e8d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5752a81d89045af8a05ff1fabd2a401 in term 0.
I20260809 02:13:44.830734 24436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c4dcacdf11644e4a8da2739866d2595" candidate_uuid: "d5752a81d89045af8a05ff1fabd2a401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f3d3c272db046b2adf30f3435b5ee8a" is_pre_election: true
I20260809 02:13:44.830861 24508 leader_election.cc:302] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4efef6fd7704b21acf3b43646e8d99b, d5752a81d89045af8a05ff1fabd2a401; no voters:
I20260809 02:13:44.830919 24436 raft_consensus.cc:2454] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5752a81d89045af8a05ff1fabd2a401 in term 0.
I20260809 02:13:44.831043 24652 raft_consensus.cc:2781] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:13:44.831108 24652 raft_consensus.cc:482] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:13:44.831143 24652 raft_consensus.cc:3037] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:44.831887 24652 raft_consensus.cc:504] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5752a81d89045af8a05ff1fabd2a401" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37217 } } peers { permanent_uuid: "c4efef6fd7704b21acf3b43646e8d99b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 37511 } } peers { permanent_uuid: "3f3d3c272db046b2adf30f3435b5ee8a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37165 } }
I20260809 02:13:44.832047 24652 leader_election.cc:288] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [CANDIDATE]: Term 1 election: Requested vote from peers c4efef6fd7704b21acf3b43646e8d99b (127.5.9.129:37511), 3f3d3c272db046b2adf30f3435b5ee8a (127.5.9.130:37165)
I20260809 02:13:44.832294 24294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c4dcacdf11644e4a8da2739866d2595" candidate_uuid: "d5752a81d89045af8a05ff1fabd2a401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4efef6fd7704b21acf3b43646e8d99b"
I20260809 02:13:44.832334 24436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c4dcacdf11644e4a8da2739866d2595" candidate_uuid: "d5752a81d89045af8a05ff1fabd2a401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f3d3c272db046b2adf30f3435b5ee8a"
I20260809 02:13:44.832408 24294 raft_consensus.cc:3037] T 9c4dcacdf11644e4a8da2739866d2595 P c4efef6fd7704b21acf3b43646e8d99b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:44.832425 24436 raft_consensus.cc:3037] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:44.833470 24436 raft_consensus.cc:2454] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5752a81d89045af8a05ff1fabd2a401 in term 1.
I20260809 02:13:44.833470 24294 raft_consensus.cc:2454] T 9c4dcacdf11644e4a8da2739866d2595 P c4efef6fd7704b21acf3b43646e8d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5752a81d89045af8a05ff1fabd2a401 in term 1.
I20260809 02:13:44.833750 24508 leader_election.cc:302] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f3d3c272db046b2adf30f3435b5ee8a, d5752a81d89045af8a05ff1fabd2a401; no voters:
I20260809 02:13:44.833894 24652 raft_consensus.cc:2781] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:44.834103 24652 raft_consensus.cc:686] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [term 1 LEADER]: Becoming Leader. State: Replica: d5752a81d89045af8a05ff1fabd2a401, State: Running, Role: LEADER
I20260809 02:13:44.834273 24652 consensus_queue.cc:237] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5752a81d89045af8a05ff1fabd2a401" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37217 } } peers { permanent_uuid: "c4efef6fd7704b21acf3b43646e8d99b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 37511 } } peers { permanent_uuid: "3f3d3c272db046b2adf30f3435b5ee8a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37165 } }
I20260809 02:13:44.835274 24137 catalog_manager.cc:5495] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5752a81d89045af8a05ff1fabd2a401 (127.5.9.131). New cstate: current_term: 1 leader_uuid: "d5752a81d89045af8a05ff1fabd2a401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5752a81d89045af8a05ff1fabd2a401" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 37217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4efef6fd7704b21acf3b43646e8d99b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 37511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f3d3c272db046b2adf30f3435b5ee8a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37165 } health_report { overall_health: UNKNOWN } } }
I20260809 02:13:44.904389 24642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.901845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56810 (local address 127.5.9.131:37217)
0809 02:13:44.901916 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:44.901918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:44.901940 (+ 22us) server_negotiation.cc:407] Connection header received
0809 02:13:44.901996 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:44.901997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:44.902017 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:44.902095 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:13:44.902249 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.902736 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.903558 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:44.903758 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:44.903831 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:44.904009 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:13:44.904097 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:13:44.904205 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:44.904301 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 02:13:44.904311 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:13:44.906307 24436 raft_consensus.cc:1261] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a [term 1 FOLLOWER]: Refusing update from remote peer d5752a81d89045af8a05ff1fabd2a401: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:13:44.906363 24294 raft_consensus.cc:1261] T 9c4dcacdf11644e4a8da2739866d2595 P c4efef6fd7704b21acf3b43646e8d99b [term 1 FOLLOWER]: Refusing update from remote peer d5752a81d89045af8a05ff1fabd2a401: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:13:44.906738 24656 consensus_queue.cc:1035] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4efef6fd7704b21acf3b43646e8d99b" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 37511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:13:44.906898 24652 consensus_queue.cc:1035] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f3d3c272db046b2adf30f3435b5ee8a" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 37165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:13:44.911598 24136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:13:44.911689 24136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:13:44.912532 24136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b06bc5abef424ed8b7f6b66c863c7e19: Sending DeleteTablet for 3 replicas of tablet 9c4dcacdf11644e4a8da2739866d2595
I20260809 02:13:44.912910 24556 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c4dcacdf11644e4a8da2739866d2595 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:44 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:56802
I20260809 02:13:44.913029 24414 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c4dcacdf11644e4a8da2739866d2595 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:44 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:34984
I20260809 02:13:44.913160 24669 tablet_replica.cc:330] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401: stopping tablet replica
I20260809 02:13:44.913278 24669 raft_consensus.cc:2229] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:13:44.913319 24273 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c4dcacdf11644e4a8da2739866d2595 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:44 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:42608
I20260809 02:13:44.913383 24669 raft_consensus.cc:2258] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:13:44.913307 24670 tablet_replica.cc:330] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a: stopping tablet replica
I20260809 02:13:44.913578 24670 raft_consensus.cc:2229] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:13:44.913677 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 24204
I20260809 02:13:44.913848 24670 raft_consensus.cc:2258] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:13:44.914374 24670 ts_tablet_manager.cc:1905] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:13:44.916170 24669 ts_tablet_manager.cc:1905] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:13:44.916210 24670 ts_tablet_manager.cc:1918] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:13:44.916429 24670 log.cc:1196] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/9c4dcacdf11644e4a8da2739866d2595
I20260809 02:13:44.916877 24670 ts_tablet_manager.cc:1939] T 9c4dcacdf11644e4a8da2739866d2595 P 3f3d3c272db046b2adf30f3435b5ee8a: Deleting consensus metadata
I20260809 02:13:44.917470 24121 catalog_manager.cc:4841] TS 3f3d3c272db046b2adf30f3435b5ee8a (127.5.9.130:37165): tablet 9c4dcacdf11644e4a8da2739866d2595 (table test-table [id=0c74e3e300924c40a1cbde22dfbaad02]) successfully deleted
I20260809 02:13:44.917825 24669 ts_tablet_manager.cc:1918] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:13:44.917904 24669 log.cc:1196] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/9c4dcacdf11644e4a8da2739866d2595
I20260809 02:13:44.918165 24669 ts_tablet_manager.cc:1939] T 9c4dcacdf11644e4a8da2739866d2595 P d5752a81d89045af8a05ff1fabd2a401: Deleting consensus metadata
I20260809 02:13:44.920562 24122 catalog_manager.cc:4841] TS d5752a81d89045af8a05ff1fabd2a401 (127.5.9.131:37217): tablet 9c4dcacdf11644e4a8da2739866d2595 (table test-table [id=0c74e3e300924c40a1cbde22dfbaad02]) successfully deleted
W20260809 02:13:44.927474 24121 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:37511 (error 108)
I20260809 02:13:44.927697 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 24348
I20260809 02:13:44.930131 24643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:44.929966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:37511 (local address 127.0.0.1:42622)
0809 02:13:44.930021 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:44.930080 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:37511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 02:13:44.930331 24121 catalog_manager.cc:4584] TS c4efef6fd7704b21acf3b43646e8d99b (127.5.9.129:37511): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9c4dcacdf11644e4a8da2739866d2595: Network error: Client connection negotiation failed: client connection to 127.5.9.129:37511: connect: Connection refused (error 111)
I20260809 02:13:44.935576 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 24491
I20260809 02:13:44.942157 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 24100
2026-08-09T02:13:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2460 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T02:13:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:13:44Z Disabled control of system clock
I20260809 02:13:44.973351 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:42609
--webserver_interface=127.5.9.190
--webserver_port=0
--builtin_ntp_servers=127.5.9.148:38271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:42609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:13:45.059418 24678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:45.059652 24678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:45.059674 24678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:45.060770 24678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:13:45.060804 24678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:45.060817 24678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:45.060829 24678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:13:45.060842 24678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:13:45.062155 24678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:38271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:42609
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:42609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:45.062368 24678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:45.062621 24678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:45.067682 24688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:45.067691 24685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:45.067755 24686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:45.067965 24678 server_base.cc:1034] running on GCE node
I20260809 02:13:45.068272 24678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:45.068569 24678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:45.069684 24678 hybrid_clock.cc:648] HybridClock initialized: now 1786241625069664 us; error 33 us; skew 500 ppm
I20260809 02:13:45.074443 24678 webserver.cc:460] Webserver started at http://127.5.9.190:33511/ using document root <none> and password file <none>
I20260809 02:13:45.074741 24678 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:45.074795 24678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:45.075009 24678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:45.076157 24678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "d7568b391fcf433cb8d9ac4c1e03c718"
format_stamp: "Formatted at 2026-08-09 02:13:45 on dist-test-slave-p3t5"
server_key: "a09b8a18f7d875e50e0f82c9abd143ea"
server_key_iv: "ff9d3e2328fc3922fee71584533e955b"
server_key_version: "encryptionkey@0"
I20260809 02:13:45.076550 24678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "d7568b391fcf433cb8d9ac4c1e03c718"
format_stamp: "Formatted at 2026-08-09 02:13:45 on dist-test-slave-p3t5"
server_key: "a09b8a18f7d875e50e0f82c9abd143ea"
server_key_iv: "ff9d3e2328fc3922fee71584533e955b"
server_key_version: "encryptionkey@0"
I20260809 02:13:45.078040 24678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:13:45.078955 24694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:45.079237 24678 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:13:45.079351 24678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "d7568b391fcf433cb8d9ac4c1e03c718"
format_stamp: "Formatted at 2026-08-09 02:13:45 on dist-test-slave-p3t5"
server_key: "a09b8a18f7d875e50e0f82c9abd143ea"
server_key_iv: "ff9d3e2328fc3922fee71584533e955b"
server_key_version: "encryptionkey@0"
I20260809 02:13:45.079440 24678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:45.092299 24678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:45.092665 24678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:45.092835 24678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:45.097319 24678 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:42609
I20260809 02:13:45.097353 24749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:42609 every 8 connection(s)
I20260809 02:13:45.097764 24678 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:13:45.098479 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 24678
I20260809 02:13:45.098570 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:13:45.098795 5158 external_mini_cluster.cc:1428] Setting key 8ab1a032ddf25fcf2425a8e381fb69c0
I20260809 02:13:45.098919 24750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:45.101928 24750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718: Bootstrap starting.
I20260809 02:13:45.102576 24750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:45.102876 24750 log.cc:824] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:45.103875 24750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718: No bootstrap required, opened a new log
I20260809 02:13:45.104547 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:45.099403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39912 (local address 127.5.9.190:42609)
0809 02:13:45.099642 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:45.099650 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:45.099676 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:13:45.100080 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:45.100086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:45.100147 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:45.100304 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:45.100564 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.101446 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.102364 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.102838 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.102996 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:45.103757 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:45.103761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:45.103761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:45.103982 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:45.104033 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:45.104045 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:45.104102 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:45.104213 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 02:13:45.104304 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":102,"threads_started":1}
I20260809 02:13:45.106117 24750 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7568b391fcf433cb8d9ac4c1e03c718" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42609 } }
I20260809 02:13:45.106254 24750 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:45.106277 24750 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7568b391fcf433cb8d9ac4c1e03c718, State: Initialized, Role: FOLLOWER
I20260809 02:13:45.106395 24750 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7568b391fcf433cb8d9ac4c1e03c718" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42609 } }
I20260809 02:13:45.106465 24750 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:13:45.106487 24750 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:13:45.106516 24750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:45.107107 24750 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7568b391fcf433cb8d9ac4c1e03c718" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42609 } }
I20260809 02:13:45.107199 24750 leader_election.cc:302] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7568b391fcf433cb8d9ac4c1e03c718; no voters:
I20260809 02:13:45.107367 24750 leader_election.cc:288] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:13:45.107465 24755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:45.107676 24755 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [term 1 LEADER]: Becoming Leader. State: Replica: d7568b391fcf433cb8d9ac4c1e03c718, State: Running, Role: LEADER
I20260809 02:13:45.107685 24750 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:13:45.107784 24755 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7568b391fcf433cb8d9ac4c1e03c718" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42609 } }
I20260809 02:13:45.108225 24756 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7568b391fcf433cb8d9ac4c1e03c718" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7568b391fcf433cb8d9ac4c1e03c718" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42609 } } }
I20260809 02:13:45.108336 24756 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:45.108206 24757 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7568b391fcf433cb8d9ac4c1e03c718. Latest consensus state: current_term: 1 leader_uuid: "d7568b391fcf433cb8d9ac4c1e03c718" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7568b391fcf433cb8d9ac4c1e03c718" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 42609 } } }
I20260809 02:13:45.108565 24757 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:45.108704 24765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:13:45.109191 24765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:13:45.110872 24765 catalog_manager.cc:1347] Generated new cluster ID: 83705349a8484918b26b984e1875594e
I20260809 02:13:45.110937 24765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:13:45.122056 24765 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:13:45.122598 24765 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:13:45.128957 24765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718: Generated new TSK 0
I20260809 02:13:45.129170 24765 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:13:45.133482 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42609
--builtin_ntp_servers=127.5.9.148:38271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:13:45.229445 24775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:45.229751 24775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:45.229780 24775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:45.230967 24775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:45.231019 24775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:45.231045 24775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:13:45.232436 24775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:38271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:45.232700 24775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:45.233034 24775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:45.239105 24780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:45.239114 24783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:45.239121 24781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:45.239584 24775 server_base.cc:1034] running on GCE node
I20260809 02:13:45.239800 24775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:45.240048 24775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:45.241225 24775 hybrid_clock.cc:648] HybridClock initialized: now 1786241625241206 us; error 35 us; skew 500 ppm
I20260809 02:13:45.242794 24775 webserver.cc:460] Webserver started at http://127.5.9.129:45313/ using document root <none> and password file <none>
I20260809 02:13:45.243072 24775 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:45.243142 24775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:45.243312 24775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:45.244449 24775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "6828d153e8794043af3a9c619490c883"
format_stamp: "Formatted at 2026-08-09 02:13:45 on dist-test-slave-p3t5"
server_key: "8385c551490019550c7e8120cb3692dc"
server_key_iv: "8d863490322defeba81e23b82262907d"
server_key_version: "encryptionkey@0"
I20260809 02:13:45.244858 24775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "6828d153e8794043af3a9c619490c883"
format_stamp: "Formatted at 2026-08-09 02:13:45 on dist-test-slave-p3t5"
server_key: "8385c551490019550c7e8120cb3692dc"
server_key_iv: "8d863490322defeba81e23b82262907d"
server_key_version: "encryptionkey@0"
I20260809 02:13:45.246376 24775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:13:45.247247 24789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:45.247432 24775 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:45.247516 24775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "6828d153e8794043af3a9c619490c883"
format_stamp: "Formatted at 2026-08-09 02:13:45 on dist-test-slave-p3t5"
server_key: "8385c551490019550c7e8120cb3692dc"
server_key_iv: "8d863490322defeba81e23b82262907d"
server_key_version: "encryptionkey@0"
I20260809 02:13:45.247596 24775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:45.266310 24775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:45.266695 24775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:45.266880 24775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:45.267160 24775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:45.267534 24775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:45.267593 24775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:45.267642 24775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:45.267673 24775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:45.273924 24775 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:39953
I20260809 02:13:45.274017 24910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:39953 every 8 connection(s)
I20260809 02:13:45.274371 24775 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:13:45.279807 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:45.275232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:47233 (local address 127.5.9.190:42609)
0809 02:13:45.275296 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:45.275298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:45.275558 (+ 260us) server_negotiation.cc:407] Connection header received
0809 02:13:45.276052 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:45.276053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:45.276077 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:45.276148 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:45.276885 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.277390 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.278449 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.278653 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.278714 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:45.279543 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:45.279544 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:45.279544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:45.279550 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:45.279564 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:45.279568 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:45.279613 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:45.279703 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 02:13:45.279740 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 02:13:45.280011 24913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:45.275177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:42609 (local address 127.5.9.129:47233)
0809 02:13:45.275392 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:45.275439 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:45.275963 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:45.276226 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:45.276230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:45.276244 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:45.276780 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:45.276784 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.277481 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.277482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:45.278361 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:45.278363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.278736 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.278736 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:45.278852 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:45.279400 (+ 548us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:13:45.279401 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:45.279430 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:13:45.279460 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:45.279633 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:45.279636 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:45.279639 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:13:45.279690 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 02:13:45.279776 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":114,"threads_started":1}
I20260809 02:13:45.280586 24911 heartbeater.cc:347] Connected to a master server at 127.5.9.190:42609
I20260809 02:13:45.280702 24911 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:45.280920 24911 heartbeater.cc:511] Master 127.5.9.190:42609 requested a full tablet report, sending...
I20260809 02:13:45.281523 24711 ts_manager.cc:194] Registered new tserver with Master: 6828d153e8794043af3a9c619490c883 (127.5.9.129:39953)
I20260809 02:13:45.281618 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 24775
I20260809 02:13:45.281764 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:13:45.281980 5158 external_mini_cluster.cc:1428] Setting key a9afef7b632a337f2654ab0ae11cb8f6
I20260809 02:13:45.282307 24711 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.9.129:47233
I20260809 02:13:45.283139 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42609
--builtin_ntp_servers=127.5.9.148:38271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:13:45.369886 24914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:45.370112 24914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:45.370137 24914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:45.371269 24914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:45.371306 24914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:45.371330 24914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:13:45.372663 24914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:38271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:45.372859 24914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:45.373116 24914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:45.378193 24922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:45.378190 24919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:45.378245 24920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:45.378727 24914 server_base.cc:1034] running on GCE node
I20260809 02:13:45.378999 24914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:45.379220 24914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:45.380360 24914 hybrid_clock.cc:648] HybridClock initialized: now 1786241625380340 us; error 39 us; skew 500 ppm
I20260809 02:13:45.381703 24914 webserver.cc:460] Webserver started at http://127.5.9.130:33949/ using document root <none> and password file <none>
I20260809 02:13:45.382036 24914 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:45.382089 24914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:45.382249 24914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:45.383437 24914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "1c4fd9423a5d49098e8c8475b90f3724"
format_stamp: "Formatted at 2026-08-09 02:13:45 on dist-test-slave-p3t5"
server_key: "dc5a222c315c0f41d2b4ee5ed16fba69"
server_key_iv: "fd9475f8e3a67a6edd7fa5c34562db7b"
server_key_version: "encryptionkey@0"
I20260809 02:13:45.383811 24914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "1c4fd9423a5d49098e8c8475b90f3724"
format_stamp: "Formatted at 2026-08-09 02:13:45 on dist-test-slave-p3t5"
server_key: "dc5a222c315c0f41d2b4ee5ed16fba69"
server_key_iv: "fd9475f8e3a67a6edd7fa5c34562db7b"
server_key_version: "encryptionkey@0"
I20260809 02:13:45.385169 24914 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:13:45.386065 24929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:45.386267 24914 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:13:45.386365 24914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "1c4fd9423a5d49098e8c8475b90f3724"
format_stamp: "Formatted at 2026-08-09 02:13:45 on dist-test-slave-p3t5"
server_key: "dc5a222c315c0f41d2b4ee5ed16fba69"
server_key_iv: "fd9475f8e3a67a6edd7fa5c34562db7b"
server_key_version: "encryptionkey@0"
I20260809 02:13:45.386445 24914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:45.395846 24914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:45.396184 24914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:45.396346 24914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:45.396601 24914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:45.397023 24914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:45.397060 24914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:45.397085 24914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:45.397135 24914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:45.403501 24914 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:40269
I20260809 02:13:45.403589 25048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:40269 every 8 connection(s)
I20260809 02:13:45.403950 24914 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:13:45.409446 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 24914
I20260809 02:13:45.409575 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:13:45.409554 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:45.404768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:60095 (local address 127.5.9.190:42609)
0809 02:13:45.404835 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:45.404837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:45.405082 (+ 245us) server_negotiation.cc:407] Connection header received
0809 02:13:45.405596 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:45.405597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:45.405607 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:45.405719 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:45.406411 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.406966 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.408026 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.408275 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.408338 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:45.409228 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:45.409230 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:45.409230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:45.409234 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:45.409250 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:45.409254 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:45.409304 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:45.409431 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 02:13:45.409456 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:13:45.409845 5158 external_mini_cluster.cc:1428] Setting key f67008061b76256bf89ec474fb459043
I20260809 02:13:45.409873 25051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:45.404743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:42609 (local address 127.5.9.130:60095)
0809 02:13:45.404938 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:45.404983 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:45.405504 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:45.405790 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:45.405793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:45.405802 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:45.406319 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:45.406324 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.407054 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.407055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:45.407940 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:45.407943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.408359 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.408360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:45.408554 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:45.409096 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:13:45.409096 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:45.409124 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:13:45.409146 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:45.409333 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:45.409339 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:45.409342 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:13:45.409427 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 02:13:45.409604 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
I20260809 02:13:45.410426 25049 heartbeater.cc:347] Connected to a master server at 127.5.9.190:42609
I20260809 02:13:45.410542 25049 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:45.410771 25049 heartbeater.cc:511] Master 127.5.9.190:42609 requested a full tablet report, sending...
I20260809 02:13:45.411029 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42609
--builtin_ntp_servers=127.5.9.148:38271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:13:45.411237 24711 ts_manager.cc:194] Registered new tserver with Master: 1c4fd9423a5d49098e8c8475b90f3724 (127.5.9.130:40269)
I20260809 02:13:45.411631 24711 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.9.130:60095
W20260809 02:13:45.500257 25052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:45.500525 25052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:45.500568 25052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:45.501832 25052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:45.501891 25052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:45.501931 25052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:13:45.503369 25052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:38271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:42609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:45.503664 25052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:45.503988 25052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:45.509503 25061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:45.509555 25059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:45.509506 25058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:45.509696 25052 server_base.cc:1034] running on GCE node
I20260809 02:13:45.510010 25052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:45.510277 25052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:45.511443 25052 hybrid_clock.cc:648] HybridClock initialized: now 1786241625511428 us; error 46 us; skew 500 ppm
I20260809 02:13:45.512907 25052 webserver.cc:460] Webserver started at http://127.5.9.131:39053/ using document root <none> and password file <none>
I20260809 02:13:45.513207 25052 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:45.513285 25052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:45.513453 25052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:45.514655 25052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "e168d44b61f14077ac01e628522a338f"
format_stamp: "Formatted at 2026-08-09 02:13:45 on dist-test-slave-p3t5"
server_key: "6a197b1f1711338a52af8a12fc154fd8"
server_key_iv: "4c1976638df9d73a8053623dce04c375"
server_key_version: "encryptionkey@0"
I20260809 02:13:45.515048 25052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "e168d44b61f14077ac01e628522a338f"
format_stamp: "Formatted at 2026-08-09 02:13:45 on dist-test-slave-p3t5"
server_key: "6a197b1f1711338a52af8a12fc154fd8"
server_key_iv: "4c1976638df9d73a8053623dce04c375"
server_key_version: "encryptionkey@0"
I20260809 02:13:45.516485 25052 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:13:45.517236 25067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:45.517403 25052 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:13:45.517517 25052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "e168d44b61f14077ac01e628522a338f"
format_stamp: "Formatted at 2026-08-09 02:13:45 on dist-test-slave-p3t5"
server_key: "6a197b1f1711338a52af8a12fc154fd8"
server_key_iv: "4c1976638df9d73a8053623dce04c375"
server_key_version: "encryptionkey@0"
I20260809 02:13:45.517601 25052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:45.552917 25052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:45.553293 25052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:45.553462 25052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:45.553789 25052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:45.554173 25052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:45.554234 25052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:45.554280 25052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:45.554307 25052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:45.560755 25052 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:39525
I20260809 02:13:45.560802 25187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:39525 every 8 connection(s)
I20260809 02:13:45.561167 25052 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:13:45.566717 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:45.561933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:35679 (local address 127.5.9.190:42609)
0809 02:13:45.561984 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:45.561986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:45.562223 (+ 237us) server_negotiation.cc:407] Connection header received
0809 02:13:45.562778 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:45.562779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:45.562792 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:45.562858 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:45.563575 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.564108 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.565136 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.565350 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.565440 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:45.566421 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:45.566422 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:45.566422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:45.566427 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:45.566443 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:45.566447 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:45.566523 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:45.566600 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 02:13:45.566629 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:13:45.566962 25191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:45.561887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:42609 (local address 127.5.9.131:35679)
0809 02:13:45.562089 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:45.562125 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:45.562665 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:45.562932 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:45.562936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:45.562949 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:45.563473 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:45.563478 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.564216 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.564218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:45.565035 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:45.565038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.565453 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.565454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:45.565596 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:45.566259 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:13:45.566260 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:45.566289 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:13:45.566313 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:45.566525 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:45.566531 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:45.566534 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:13:45.566605 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 02:13:45.566709 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260809 02:13:45.567400 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 25052
I20260809 02:13:45.567538 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:13:45.567556 25188 heartbeater.cc:347] Connected to a master server at 127.5.9.190:42609
I20260809 02:13:45.567649 25188 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:45.567772 5158 external_mini_cluster.cc:1428] Setting key 403351353d3b19a07885a038d63f65f2
I20260809 02:13:45.567863 25188 heartbeater.cc:511] Master 127.5.9.190:42609 requested a full tablet report, sending...
I20260809 02:13:45.568343 24711 ts_manager.cc:194] Registered new tserver with Master: e168d44b61f14077ac01e628522a338f (127.5.9.131:39525)
I20260809 02:13:45.568738 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:13:45.568856 24711 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.9.131:35679
I20260809 02:13:45.573290 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:45.570282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35642 (local address 127.5.9.190:42609)
0809 02:13:45.570323 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:45.570325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:45.570479 (+ 154us) server_negotiation.cc:407] Connection header received
0809 02:13:45.570546 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:45.570547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:45.570557 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:45.570631 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:45.570821 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.571284 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.572048 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.572241 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.572323 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:45.572990 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:45.572991 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:45.572991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:45.572996 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:45.573018 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:45.573021 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:45.573121 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:45.573193 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 02:13:45.573204 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:13:45.575014 24711 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:13:45.576623 24711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:13:45.583361 25202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:45.579039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49320 (local address 127.5.9.131:39525)
0809 02:13:45.579227 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:45.579232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:45.579311 (+ 79us) server_negotiation.cc:407] Connection header received
0809 02:13:45.579404 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:45.579406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:45.579423 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:45.579771 (+ 348us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:45.579825 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.580945 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.582179 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.582942 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.583028 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:45.583128 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:45.583208 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 02:13:45.583249 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
I20260809 02:13:45.583338 25203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:45.578873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:39525 (local address 127.0.0.1:49320)
0809 02:13:45.579247 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:45.579259 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:45.579325 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:45.579605 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:45.579610 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:45.579613 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:45.579693 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:45.579695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.581039 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.581041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:45.582084 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:45.582087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.583028 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.583029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:45.583108 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:45.583123 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 02:13:45.583197 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 02:13:45.583249 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":2,"thread_start_us":190,"threads_started":1}
I20260809 02:13:45.583472 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:45.579109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:39953 (local address 127.0.0.1:39326)
0809 02:13:45.579354 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:45.579364 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:45.579400 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:45.579712 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:45.579712 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:45.579715 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:45.579763 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:45.579765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.580670 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.580671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:45.582044 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:45.582047 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.583199 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.583200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:45.583261 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:45.583266 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:13:45.583317 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 02:13:45.583359 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":41,"threads_started":1}
I20260809 02:13:45.583720 25204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:45.579291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39326 (local address 127.5.9.129:39953)
0809 02:13:45.579460 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:45.579464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:45.579474 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:13:45.579519 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:45.579521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:45.579533 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:45.579633 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:45.579848 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.580572 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.582154 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.583029 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.583088 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:45.583161 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:45.583587 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 02:13:45.583639 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260809 02:13:45.584292 25118 tablet_service.cc:1467] Processing CreateTablet for tablet 20438f2fef8f40b69a70568f6b01c1eb (DEFAULT_TABLE table=test-table [id=cb84fbf2eda84ad580c931b9bc3d2472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:45.584438 24839 tablet_service.cc:1467] Processing CreateTablet for tablet 20438f2fef8f40b69a70568f6b01c1eb (DEFAULT_TABLE table=test-table [id=cb84fbf2eda84ad580c931b9bc3d2472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:45.584704 25118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20438f2fef8f40b69a70568f6b01c1eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:45.584815 25200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:45.578873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:40269 (local address 127.0.0.1:33440)
0809 02:13:45.579143 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:45.579187 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:45.579258 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:45.579726 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:45.579727 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:45.579729 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:45.579767 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:45.579769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.582173 (+ 2404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.582174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:45.582866 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:45.582868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.584548 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.584549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:45.584617 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:45.584621 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:13:45.584687 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 02:13:45.584728 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":86,"threads_started":1}
I20260809 02:13:45.584824 25201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:45.578972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33440 (local address 127.5.9.130:40269)
0809 02:13:45.579521 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:45.579525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:45.579534 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:13:45.579572 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:45.579574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:45.579590 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:45.579658 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:45.579842 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.580956 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.582960 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:45.584424 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:45.584520 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:45.584611 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:45.584696 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 02:13:45.584743 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":96,"threads_started":1}
I20260809 02:13:45.584824 24839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20438f2fef8f40b69a70568f6b01c1eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:45.585495 24979 tablet_service.cc:1467] Processing CreateTablet for tablet 20438f2fef8f40b69a70568f6b01c1eb (DEFAULT_TABLE table=test-table [id=cb84fbf2eda84ad580c931b9bc3d2472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:45.585850 24979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20438f2fef8f40b69a70568f6b01c1eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:45.587724 25205 tablet_bootstrap.cc:492] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f: Bootstrap starting.
I20260809 02:13:45.588289 25206 tablet_bootstrap.cc:492] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883: Bootstrap starting.
I20260809 02:13:45.588680 25205 tablet_bootstrap.cc:654] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:45.588941 25206 tablet_bootstrap.cc:654] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:45.589145 25207 tablet_bootstrap.cc:492] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724: Bootstrap starting.
I20260809 02:13:45.589035 25205 log.cc:824] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:45.589289 25206 log.cc:824] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:45.590005 25207 tablet_bootstrap.cc:654] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:45.590230 25205 tablet_bootstrap.cc:492] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f: No bootstrap required, opened a new log
I20260809 02:13:45.590276 25206 tablet_bootstrap.cc:492] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883: No bootstrap required, opened a new log
I20260809 02:13:45.590337 25206 ts_tablet_manager.cc:1397] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:13:45.590345 25207 log.cc:824] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:45.590364 25205 ts_tablet_manager.cc:1397] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:13:45.591212 25207 tablet_bootstrap.cc:492] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724: No bootstrap required, opened a new log
I20260809 02:13:45.591293 25207 ts_tablet_manager.cc:1397] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:13:45.592957 25205 raft_consensus.cc:348] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4fd9423a5d49098e8c8475b90f3724" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40269 } } peers { permanent_uuid: "e168d44b61f14077ac01e628522a338f" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39525 } } peers { permanent_uuid: "6828d153e8794043af3a9c619490c883" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 39953 } }
I20260809 02:13:45.592957 25206 raft_consensus.cc:348] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4fd9423a5d49098e8c8475b90f3724" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40269 } } peers { permanent_uuid: "e168d44b61f14077ac01e628522a338f" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39525 } } peers { permanent_uuid: "6828d153e8794043af3a9c619490c883" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 39953 } }
I20260809 02:13:45.593135 25205 raft_consensus.cc:374] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:45.593135 25206 raft_consensus.cc:374] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:45.593173 25205 raft_consensus.cc:729] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e168d44b61f14077ac01e628522a338f, State: Initialized, Role: FOLLOWER
I20260809 02:13:45.593173 25206 raft_consensus.cc:729] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6828d153e8794043af3a9c619490c883, State: Initialized, Role: FOLLOWER
I20260809 02:13:45.593322 25206 consensus_queue.cc:260] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4fd9423a5d49098e8c8475b90f3724" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40269 } } peers { permanent_uuid: "e168d44b61f14077ac01e628522a338f" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39525 } } peers { permanent_uuid: "6828d153e8794043af3a9c619490c883" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 39953 } }
I20260809 02:13:45.593322 25205 consensus_queue.cc:260] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4fd9423a5d49098e8c8475b90f3724" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40269 } } peers { permanent_uuid: "e168d44b61f14077ac01e628522a338f" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39525 } } peers { permanent_uuid: "6828d153e8794043af3a9c619490c883" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 39953 } }
I20260809 02:13:45.593320 25207 raft_consensus.cc:348] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4fd9423a5d49098e8c8475b90f3724" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40269 } } peers { permanent_uuid: "e168d44b61f14077ac01e628522a338f" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39525 } } peers { permanent_uuid: "6828d153e8794043af3a9c619490c883" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 39953 } }
I20260809 02:13:45.593501 25207 raft_consensus.cc:374] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:45.593550 25207 raft_consensus.cc:729] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c4fd9423a5d49098e8c8475b90f3724, State: Initialized, Role: FOLLOWER
I20260809 02:13:45.593597 25206 ts_tablet_manager.cc:1428] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 02:13:45.593597 25205 ts_tablet_manager.cc:1428] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:13:45.593758 24911 heartbeater.cc:503] Master 127.5.9.190:42609 was elected leader, sending a full tablet report...
I20260809 02:13:45.593721 25207 consensus_queue.cc:260] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4fd9423a5d49098e8c8475b90f3724" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40269 } } peers { permanent_uuid: "e168d44b61f14077ac01e628522a338f" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39525 } } peers { permanent_uuid: "6828d153e8794043af3a9c619490c883" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 39953 } }
I20260809 02:13:45.594002 25207 ts_tablet_manager.cc:1428] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:13:45.594035 25188 heartbeater.cc:503] Master 127.5.9.190:42609 was elected leader, sending a full tablet report...
I20260809 02:13:45.594076 25049 heartbeater.cc:503] Master 127.5.9.190:42609 was elected leader, sending a full tablet report...
W20260809 02:13:45.654609 25050 tablet.cc:2369] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:13:45.775147 24912 tablet.cc:2369] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:13:45.811764 25189 tablet.cc:2369] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:13:46.005630 25211 raft_consensus.cc:482] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:13:46.005836 25211 raft_consensus.cc:504] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4fd9423a5d49098e8c8475b90f3724" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40269 } } peers { permanent_uuid: "e168d44b61f14077ac01e628522a338f" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39525 } } peers { permanent_uuid: "6828d153e8794043af3a9c619490c883" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 39953 } }
I20260809 02:13:46.006263 25211 leader_election.cc:288] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c4fd9423a5d49098e8c8475b90f3724 (127.5.9.130:40269), e168d44b61f14077ac01e628522a338f (127.5.9.131:39525)
I20260809 02:13:46.008718 25212 raft_consensus.cc:482] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:13:46.008849 25212 raft_consensus.cc:504] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4fd9423a5d49098e8c8475b90f3724" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40269 } } peers { permanent_uuid: "e168d44b61f14077ac01e628522a338f" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39525 } } peers { permanent_uuid: "6828d153e8794043af3a9c619490c883" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 39953 } }
I20260809 02:13:46.009230 25212 leader_election.cc:288] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c4fd9423a5d49098e8c8475b90f3724 (127.5.9.130:40269), 6828d153e8794043af3a9c619490c883 (127.5.9.129:39953)
I20260809 02:13:46.009949 25201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:46.006557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:41165 (local address 127.5.9.130:40269)
0809 02:13:46.006626 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:46.006628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:46.006736 (+ 108us) server_negotiation.cc:407] Connection header received
0809 02:13:46.006811 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:46.006812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:46.006825 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:46.006884 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:46.007116 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.007688 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.008942 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.009486 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.009536 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:46.009548 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:46.009814 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 02:13:46.009863 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:13:46.010032 25202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:46.006596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:33387 (local address 127.5.9.131:39525)
0809 02:13:46.006644 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:46.006646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:46.006759 (+ 113us) server_negotiation.cc:407] Connection header received
0809 02:13:46.006858 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:46.006859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:46.006886 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:46.006946 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:46.007182 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.007692 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.008995 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.009579 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.009696 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:46.009707 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:46.009911 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 02:13:46.009954 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:13:46.010116 25214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:46.006491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:40269 (local address 127.5.9.129:41165)
0809 02:13:46.006641 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:46.006653 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:46.006746 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:46.006909 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:46.006910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:46.006913 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:46.007001 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:46.007003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.007769 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.007771 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:46.008833 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:46.008836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.009632 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.009633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:46.009712 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:46.009716 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:13:46.009794 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 02:13:46.010029 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
I20260809 02:13:46.010439 25000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20438f2fef8f40b69a70568f6b01c1eb" candidate_uuid: "6828d153e8794043af3a9c619490c883" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c4fd9423a5d49098e8c8475b90f3724" is_pre_election: true
I20260809 02:13:46.010639 25000 raft_consensus.cc:2454] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6828d153e8794043af3a9c619490c883 in term 0.
I20260809 02:13:46.010885 24790 leader_election.cc:302] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c4fd9423a5d49098e8c8475b90f3724, 6828d153e8794043af3a9c619490c883; no voters:
I20260809 02:13:46.010277 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:46.006513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:39525 (local address 127.5.9.129:33387)
0809 02:13:46.006646 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:46.006657 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:46.006767 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:46.006980 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:46.006981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:46.006983 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:46.007070 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:46.007072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.007795 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.007796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:46.008848 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:46.008852 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.009743 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.009745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:46.009825 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:46.009831 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:13:46.010130 (+ 299us) client_negotiation.cc:240] Negotiation successful
0809 02:13:46.010177 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":62,"threads_started":1}
I20260809 02:13:46.011018 25211 raft_consensus.cc:2781] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:13:46.011085 25211 raft_consensus.cc:482] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:13:46.011116 25211 raft_consensus.cc:3037] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:46.011543 25140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20438f2fef8f40b69a70568f6b01c1eb" candidate_uuid: "6828d153e8794043af3a9c619490c883" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e168d44b61f14077ac01e628522a338f" is_pre_election: true
I20260809 02:13:46.011643 25140 raft_consensus.cc:2454] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6828d153e8794043af3a9c619490c883 in term 0.
I20260809 02:13:46.012115 25211 raft_consensus.cc:504] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4fd9423a5d49098e8c8475b90f3724" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40269 } } peers { permanent_uuid: "e168d44b61f14077ac01e628522a338f" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39525 } } peers { permanent_uuid: "6828d153e8794043af3a9c619490c883" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 39953 } }
I20260809 02:13:46.012266 25211 leader_election.cc:288] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [CANDIDATE]: Term 1 election: Requested vote from peers 1c4fd9423a5d49098e8c8475b90f3724 (127.5.9.130:40269), e168d44b61f14077ac01e628522a338f (127.5.9.131:39525)
I20260809 02:13:46.012421 25140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20438f2fef8f40b69a70568f6b01c1eb" candidate_uuid: "6828d153e8794043af3a9c619490c883" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e168d44b61f14077ac01e628522a338f"
I20260809 02:13:46.012477 25000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20438f2fef8f40b69a70568f6b01c1eb" candidate_uuid: "6828d153e8794043af3a9c619490c883" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c4fd9423a5d49098e8c8475b90f3724"
I20260809 02:13:46.012492 25140 raft_consensus.cc:3037] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:46.012544 25000 raft_consensus.cc:3037] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:46.013411 25140 raft_consensus.cc:2454] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6828d153e8794043af3a9c619490c883 in term 1.
I20260809 02:13:46.013411 25000 raft_consensus.cc:2454] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6828d153e8794043af3a9c619490c883 in term 1.
I20260809 02:13:46.013515 25191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:46.009388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:40269 (local address 127.5.9.131:34427)
0809 02:13:46.009652 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:46.009663 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:46.009700 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:46.010417 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:46.010418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:46.010420 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:46.010494 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:46.010495 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.011350 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.011351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:46.012496 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:46.012499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.013302 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.013302 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:46.013366 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:46.013370 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:13:46.013400 (+ 30us) client_negotiation.cc:240] Negotiation successful
0809 02:13:46.013443 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":30}
I20260809 02:13:46.013515 25204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:46.009844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:33289 (local address 127.5.9.129:39953)
0809 02:13:46.009884 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:46.009885 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:46.009891 (+ 6us) server_negotiation.cc:407] Connection header received
0809 02:13:46.009909 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:46.009910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:46.009918 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:46.009963 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:46.010434 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.011113 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.012606 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.013128 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.013172 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:46.013181 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:46.013408 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 02:13:46.013443 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:13:46.013583 24790 leader_election.cc:302] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c4fd9423a5d49098e8c8475b90f3724, 6828d153e8794043af3a9c619490c883; no voters:
I20260809 02:13:46.013738 25217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:46.009500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:39953 (local address 127.5.9.131:33289)
0809 02:13:46.009667 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:46.009678 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:46.009727 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:46.010286 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:46.010288 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:46.010291 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:46.010355 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:46.010356 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.011195 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.011195 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:46.012499 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:46.012500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.013247 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.013247 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:46.013323 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:46.013327 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:13:46.013584 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 02:13:46.013655 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":5,"thread_start_us":91,"threads_started":1}
I20260809 02:13:46.013954 25211 raft_consensus.cc:2781] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:46.013921 25216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:46.009461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:34427 (local address 127.5.9.130:40269)
0809 02:13:46.010198 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:46.010200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:46.010207 (+ 7us) server_negotiation.cc:407] Connection header received
0809 02:13:46.010227 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:46.010228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:46.010321 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:46.010559 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:46.010591 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.011265 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.012605 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.013217 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.013710 (+ 493us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:46.013723 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:46.013743 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 02:13:46.013868 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":62,"threads_started":1}
I20260809 02:13:46.014113 25211 raft_consensus.cc:686] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [term 1 LEADER]: Becoming Leader. State: Replica: 6828d153e8794043af3a9c619490c883, State: Running, Role: LEADER
I20260809 02:13:46.014098 24859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20438f2fef8f40b69a70568f6b01c1eb" candidate_uuid: "e168d44b61f14077ac01e628522a338f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6828d153e8794043af3a9c619490c883" is_pre_election: true
I20260809 02:13:46.014200 25000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20438f2fef8f40b69a70568f6b01c1eb" candidate_uuid: "e168d44b61f14077ac01e628522a338f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c4fd9423a5d49098e8c8475b90f3724" is_pre_election: true
I20260809 02:13:46.014281 25000 raft_consensus.cc:2379] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e168d44b61f14077ac01e628522a338f in current term 1: Already voted for candidate 6828d153e8794043af3a9c619490c883 in this term.
I20260809 02:13:46.014266 25211 consensus_queue.cc:237] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4fd9423a5d49098e8c8475b90f3724" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40269 } } peers { permanent_uuid: "e168d44b61f14077ac01e628522a338f" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39525 } } peers { permanent_uuid: "6828d153e8794043af3a9c619490c883" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 39953 } }
I20260809 02:13:46.014963 25068 leader_election.cc:302] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e168d44b61f14077ac01e628522a338f; no voters: 1c4fd9423a5d49098e8c8475b90f3724, 6828d153e8794043af3a9c619490c883
I20260809 02:13:46.015102 25212 raft_consensus.cc:2726] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 02:13:46.015199 24709 catalog_manager.cc:5495] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6828d153e8794043af3a9c619490c883 (127.5.9.129). New cstate: current_term: 1 leader_uuid: "6828d153e8794043af3a9c619490c883" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4fd9423a5d49098e8c8475b90f3724" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e168d44b61f14077ac01e628522a338f" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6828d153e8794043af3a9c619490c883" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 39953 } health_report { overall_health: HEALTHY } } }
I20260809 02:13:46.026654 25204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:46.024267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39342 (local address 127.5.9.129:39953)
0809 02:13:46.024330 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:46.024332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:46.024341 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:13:46.024394 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:46.024395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:46.024413 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:46.024479 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:13:46.024608 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.025078 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.025946 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:46.026134 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:46.026188 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:46.026314 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:13:46.026402 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:13:46.026478 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:46.026519 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 02:13:46.026528 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:13:46.028497 25000 raft_consensus.cc:1261] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724 [term 1 FOLLOWER]: Refusing update from remote peer 6828d153e8794043af3a9c619490c883: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:13:46.028530 25140 raft_consensus.cc:1261] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f [term 1 FOLLOWER]: Refusing update from remote peer 6828d153e8794043af3a9c619490c883: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:13:46.028874 25218 consensus_queue.cc:1035] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c4fd9423a5d49098e8c8475b90f3724" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 40269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:13:46.029047 25211 consensus_queue.cc:1035] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e168d44b61f14077ac01e628522a338f" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 39525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:13:46.031867 25222 mvcc.cc:205] Tried to move back new op lower bound from 7316445700209905664 to 7316445700154945536. Current Snapshot: MvccSnapshot[applied={T|T < 7316445700209905664}]
I20260809 02:13:46.032228 25223 mvcc.cc:205] Tried to move back new op lower bound from 7316445700209905664 to 7316445700154945536. Current Snapshot: MvccSnapshot[applied={T|T < 7316445700209905664}]
I20260809 02:13:46.034969 24710 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:13:46.035066 24710 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:13:46.035883 24710 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d7568b391fcf433cb8d9ac4c1e03c718: Sending DeleteTablet for 3 replicas of tablet 20438f2fef8f40b69a70568f6b01c1eb
I20260809 02:13:46.036223 24839 tablet_service.cc:1514] Processing DeleteTablet for tablet 20438f2fef8f40b69a70568f6b01c1eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:46 UTC) from {username='slave'} at 127.0.0.1:39326
I20260809 02:13:46.036262 24979 tablet_service.cc:1514] Processing DeleteTablet for tablet 20438f2fef8f40b69a70568f6b01c1eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:46 UTC) from {username='slave'} at 127.0.0.1:33440
I20260809 02:13:46.036556 25232 tablet_replica.cc:330] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883: stopping tablet replica
I20260809 02:13:46.036597 25118 tablet_service.cc:1514] Processing DeleteTablet for tablet 20438f2fef8f40b69a70568f6b01c1eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:46 UTC) from {username='slave'} at 127.0.0.1:49320
I20260809 02:13:46.036664 25232 raft_consensus.cc:2229] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:13:46.036756 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 24775
I20260809 02:13:46.036793 25232 raft_consensus.cc:2258] T 20438f2fef8f40b69a70568f6b01c1eb P 6828d153e8794043af3a9c619490c883 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:13:46.036902 25233 tablet_replica.cc:330] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f: stopping tablet replica
I20260809 02:13:46.037027 25233 raft_consensus.cc:2229] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:13:46.037149 25233 raft_consensus.cc:2258] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:13:46.038478 25233 ts_tablet_manager.cc:1905] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:13:46.038831 25231 tablet_replica.cc:330] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724: stopping tablet replica
I20260809 02:13:46.038951 25231 raft_consensus.cc:2229] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:13:46.039067 25231 raft_consensus.cc:2258] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:13:46.039592 25231 ts_tablet_manager.cc:1905] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:13:46.040800 25233 ts_tablet_manager.cc:1918] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:13:46.040886 25233 log.cc:1196] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/20438f2fef8f40b69a70568f6b01c1eb
I20260809 02:13:46.041155 25233 ts_tablet_manager.cc:1939] T 20438f2fef8f40b69a70568f6b01c1eb P e168d44b61f14077ac01e628522a338f: Deleting consensus metadata
I20260809 02:13:46.041378 25231 ts_tablet_manager.cc:1918] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:13:46.041476 25231 log.cc:1196] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/20438f2fef8f40b69a70568f6b01c1eb
I20260809 02:13:46.041604 24696 catalog_manager.cc:4841] TS e168d44b61f14077ac01e628522a338f (127.5.9.131:39525): tablet 20438f2fef8f40b69a70568f6b01c1eb (table test-table [id=cb84fbf2eda84ad580c931b9bc3d2472]) successfully deleted
I20260809 02:13:46.041837 25231 ts_tablet_manager.cc:1939] T 20438f2fef8f40b69a70568f6b01c1eb P 1c4fd9423a5d49098e8c8475b90f3724: Deleting consensus metadata
I20260809 02:13:46.042239 24695 catalog_manager.cc:4841] TS 1c4fd9423a5d49098e8c8475b90f3724 (127.5.9.130:40269): tablet 20438f2fef8f40b69a70568f6b01c1eb (table test-table [id=cb84fbf2eda84ad580c931b9bc3d2472]) successfully deleted
W20260809 02:13:46.044831 24695 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:39953 (error 108)
I20260809 02:13:46.045166 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 24914
I20260809 02:13:46.045414 25200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:46.045247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:39953 (local address 127.0.0.1:39354)
0809 02:13:46.045327 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:46.045373 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:39953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 02:13:46.053840 24695 catalog_manager.cc:4584] TS 6828d153e8794043af3a9c619490c883 (127.5.9.129:39953): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 20438f2fef8f40b69a70568f6b01c1eb: Network error: Client connection negotiation failed: client connection to 127.5.9.129:39953: connect: Connection refused (error 111)
I20260809 02:13:46.054291 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 25052
I20260809 02:13:46.060599 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 24678
2026-08-09T02:13:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1117 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:13:46 dist-test-slave-p3t5 krb5kdc[25237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:13:46 dist-test-slave-p3t5 krb5kdc[25237](info): set up 2 sockets
Aug 09 02:13:46 dist-test-slave-p3t5 krb5kdc[25237](info): commencing operation
krb5kdc: starting...
W20260809 02:13:47.134639 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:13:47 dist-test-slave-p3t5 krb5kdc[25237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:13:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:13:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:13:47.269793 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:40999
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:40477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:40999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:47.356045 25251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:47.356313 25251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:47.356352 25251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:47.357570 25251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:13:47.357679 25251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:47.357734 25251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:47.357764 25251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:13:47.357789 25251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:13:47.359372 25251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:40477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:40999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:40999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:47.359676 25251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:47.359935 25251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:47.365155 25259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:47.365113 25256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:47.365121 25257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:47.365578 25251 server_base.cc:1034] running on GCE node
I20260809 02:13:47.365970 25251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:47.366281 25251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:47.367456 25251 hybrid_clock.cc:648] HybridClock initialized: now 1786241627367435 us; error 41 us; skew 500 ppm
Aug 09 02:13:47 dist-test-slave-p3t5 krb5kdc[25237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241627, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:47.370265 25251 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:13:47.370797 25251 webserver.cc:460] Webserver started at http://127.5.9.190:41143/ using document root <none> and password file <none>
I20260809 02:13:47.371001 25251 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:47.371043 25251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:47.371137 25251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:47.372201 25251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "d6d2ede22e0b47c09169d184badff524"
format_stamp: "Formatted at 2026-08-09 02:13:47 on dist-test-slave-p3t5"
server_key: "07a71964bc1140251bf80254b1c14763"
server_key_iv: "73e2d4240d8b049cc13d5cf22773adfd"
server_key_version: "encryptionkey@0"
I20260809 02:13:47.372603 25251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "d6d2ede22e0b47c09169d184badff524"
format_stamp: "Formatted at 2026-08-09 02:13:47 on dist-test-slave-p3t5"
server_key: "07a71964bc1140251bf80254b1c14763"
server_key_iv: "73e2d4240d8b049cc13d5cf22773adfd"
server_key_version: "encryptionkey@0"
I20260809 02:13:47.374063 25251 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:13:47.374789 25268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:47.374991 25251 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:47.375084 25251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "d6d2ede22e0b47c09169d184badff524"
format_stamp: "Formatted at 2026-08-09 02:13:47 on dist-test-slave-p3t5"
server_key: "07a71964bc1140251bf80254b1c14763"
server_key_iv: "73e2d4240d8b049cc13d5cf22773adfd"
server_key_version: "encryptionkey@0"
I20260809 02:13:47.375144 25251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:47.387392 25251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:47.389863 25251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:47.390002 25251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:47.394550 25251 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:40999
I20260809 02:13:47.394590 25324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:40999 every 8 connection(s)
I20260809 02:13:47.395015 25251 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:13:47.396034 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 25251
I20260809 02:13:47.396118 25325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:47.396142 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:13:47.396411 5158 external_mini_cluster.cc:1428] Setting key 2d8d334e963b6a0f31d2287e9beb6d49
I20260809 02:13:47.399868 25325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524: Bootstrap starting.
I20260809 02:13:47.400650 25325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:47.400966 25325 log.cc:824] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:47.401943 25325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524: No bootstrap required, opened a new log
Aug 09 02:13:47 dist-test-slave-p3t5 krb5kdc[25237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:13:47.404632 25325 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6d2ede22e0b47c09169d184badff524" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 40999 } }
I20260809 02:13:47.404814 25325 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:47.404865 25325 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6d2ede22e0b47c09169d184badff524, State: Initialized, Role: FOLLOWER
I20260809 02:13:47.405005 25325 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6d2ede22e0b47c09169d184badff524" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 40999 } }
I20260809 02:13:47.405084 25325 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:13:47.405107 25325 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:13:47.405174 25325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:47.405905 25325 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6d2ede22e0b47c09169d184badff524" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 40999 } }
I20260809 02:13:47.406064 25325 leader_election.cc:302] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6d2ede22e0b47c09169d184badff524; no voters:
I20260809 02:13:47.406293 25325 leader_election.cc:288] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:13:47.406368 25330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:47.406584 25330 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [term 1 LEADER]: Becoming Leader. State: Replica: d6d2ede22e0b47c09169d184badff524, State: Running, Role: LEADER
I20260809 02:13:47.406620 25325 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:13:47.406702 25330 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6d2ede22e0b47c09169d184badff524" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 40999 } }
I20260809 02:13:47.407232 25331 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6d2ede22e0b47c09169d184badff524. Latest consensus state: current_term: 1 leader_uuid: "d6d2ede22e0b47c09169d184badff524" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6d2ede22e0b47c09169d184badff524" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 40999 } } }
I20260809 02:13:47.407347 25331 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:47.407238 25332 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6d2ede22e0b47c09169d184badff524" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6d2ede22e0b47c09169d184badff524" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 40999 } } }
I20260809 02:13:47.407539 25332 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524 [sys.catalog]: This master's current role is: LEADER
W20260809 02:13:47.408743 25345 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:13:47.408855 25345 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:13:47.408972 25339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:13:47.409240 25339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:13:47.409603 25328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:47.397071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58336 (local address 127.5.9.190:40999)
0809 02:13:47.397286 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:47.397296 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:47.397329 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:13:47.397951 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:47.397961 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:47.398033 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:47.398197 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:47.398911 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.400333 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.401249 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.401801 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.401940 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:47.404865 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:47.404870 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:47.404870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:47.404886 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:47.407415 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:47.407943 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:47.407946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:47.407947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:47.408077 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:47.408315 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:47.408316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:47.408316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:47.408650 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:47.408804 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:47.409225 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 02:13:47.409392 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":68,"threads_started":1}
I20260809 02:13:47.411203 25339 catalog_manager.cc:1347] Generated new cluster ID: 817a2ee672aa46e28987e7ee6a45c75a
I20260809 02:13:47.411270 25339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:13:47.425390 25339 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:13:47.426190 25339 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:13:47.429663 25339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524: Generated new TSK 0
I20260809 02:13:47.429895 25339 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.9.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
Entry for principal kudu/127.5.9.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab.
I20260809 02:13:47.464835 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:40999
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:40477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:47.552362 25351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:47.552605 25351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:47.552634 25351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:47.553898 25351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:47.553944 25351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:47.553973 25351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:13:47.555505 25351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:40477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.129.keytab
--principal=kudu/127.5.9.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:47.555771 25351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:47.556036 25351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:47.561367 25360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:47.561321 25356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:47.561321 25358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:47.561807 25351 server_base.cc:1034] running on GCE node
I20260809 02:13:47.561987 25351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:47.562193 25351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:47.563337 25351 hybrid_clock.cc:648] HybridClock initialized: now 1786241627563324 us; error 33 us; skew 500 ppm
Aug 09 02:13:47 dist-test-slave-p3t5 krb5kdc[25237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241627, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:47.566255 25351 init.cc:377] Logged in from keytab as kudu/127.5.9.129@KRBTEST.COM (short username kudu)
I20260809 02:13:47.566747 25351 webserver.cc:460] Webserver started at http://127.5.9.129:36433/ using document root <none> and password file <none>
I20260809 02:13:47.566995 25351 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:47.567044 25351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:47.567211 25351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:47.568336 25351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "83f528b9721b42b58fe95216cd85a36e"
format_stamp: "Formatted at 2026-08-09 02:13:47 on dist-test-slave-p3t5"
server_key: "0fdf54571eb9ba3d8bd86a3c9aa1138f"
server_key_iv: "c8e480759c4d5b6fe1de844084628067"
server_key_version: "encryptionkey@0"
I20260809 02:13:47.568718 25351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "83f528b9721b42b58fe95216cd85a36e"
format_stamp: "Formatted at 2026-08-09 02:13:47 on dist-test-slave-p3t5"
server_key: "0fdf54571eb9ba3d8bd86a3c9aa1138f"
server_key_iv: "c8e480759c4d5b6fe1de844084628067"
server_key_version: "encryptionkey@0"
I20260809 02:13:47.570317 25351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:13:47.571228 25367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:47.571491 25351 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:47.571571 25351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "83f528b9721b42b58fe95216cd85a36e"
format_stamp: "Formatted at 2026-08-09 02:13:47 on dist-test-slave-p3t5"
server_key: "0fdf54571eb9ba3d8bd86a3c9aa1138f"
server_key_iv: "c8e480759c4d5b6fe1de844084628067"
server_key_version: "encryptionkey@0"
I20260809 02:13:47.571682 25351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:47.580425 25351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:47.583021 25351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:47.583201 25351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:47.583482 25351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:47.583820 25351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:47.583876 25351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:47.583911 25351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:47.583941 25351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:47.590144 25351 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:46465
I20260809 02:13:47.590209 25492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:46465 every 8 connection(s)
I20260809 02:13:47.590605 25351 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:13:47.591132 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 25351
I20260809 02:13:47.591264 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:13:47.591534 5158 external_mini_cluster.cc:1428] Setting key 25f57e7d34939017a1f24016b08b39a5
Aug 09 02:13:47 dist-test-slave-p3t5 krb5kdc[25237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241627, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.129@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.9.130@KRBTEST.COM; defaulting to no policy
I20260809 02:13:47.602384 25328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:47.591490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:39087 (local address 127.5.9.190:40999)
0809 02:13:47.591572 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:47.591574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:47.591823 (+ 249us) server_negotiation.cc:407] Connection header received
0809 02:13:47.592495 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:47.592496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:47.592506 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:47.592547 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:47.594053 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.594718 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.596187 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.596424 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.596525 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:47.599061 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:47.599063 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:47.599063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:47.599072 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:47.600670 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:47.601163 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:47.601165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:47.601165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:47.601239 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:47.601525 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:47.601526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:47.601526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:47.601728 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:47.601811 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:47.602167 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 02:13:47.602272 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:13:47.602543 25496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:47.591446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:40999 (local address 127.5.9.129:39087)
0809 02:13:47.591675 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:47.591731 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:47.592380 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:47.592642 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:47.592647 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:47.593558 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:47.593918 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:47.593927 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.594836 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.594837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:47.596082 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:47.596086 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.596541 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.596542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:47.596692 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:47.597306 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:13:47.597315 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:47.598952 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:47.600798 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:47.600808 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:47.600811 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:47.601077 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:47.601321 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:47.601321 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:47.601322 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:47.601446 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:47.601814 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:47.601823 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:47.601987 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 02:13:47.602102 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 02:13:47.602293 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":108,"threads_started":1}
I20260809 02:13:47.603035 25493 heartbeater.cc:347] Connected to a master server at 127.5.9.190:40999
I20260809 02:13:47.603139 25493 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:47.603365 25493 heartbeater.cc:511] Master 127.5.9.190:40999 requested a full tablet report, sending...
I20260809 02:13:47.603924 25286 ts_manager.cc:194] Registered new tserver with Master: 83f528b9721b42b58fe95216cd85a36e (127.5.9.129:46465)
I20260809 02:13:47.604518 25286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.129@KRBTEST.COM'} at 127.5.9.129:39087
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
Entry for principal kudu/127.5.9.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab.
I20260809 02:13:47.623309 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:40999
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:40477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:47.709836 25499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:47.710083 25499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:47.710110 25499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:47.711356 25499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:47.711406 25499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:47.711434 25499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:13:47.713023 25499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:40477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.130.keytab
--principal=kudu/127.5.9.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:47.713292 25499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:47.713547 25499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:47.719046 25505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:47.719058 25507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:47.719077 25504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:47.719523 25499 server_base.cc:1034] running on GCE node
I20260809 02:13:47.719727 25499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:47.719985 25499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:47.721131 25499 hybrid_clock.cc:648] HybridClock initialized: now 1786241627721117 us; error 30 us; skew 500 ppm
Aug 09 02:13:47 dist-test-slave-p3t5 krb5kdc[25237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241627, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:47.723814 25499 init.cc:377] Logged in from keytab as kudu/127.5.9.130@KRBTEST.COM (short username kudu)
I20260809 02:13:47.724278 25499 webserver.cc:460] Webserver started at http://127.5.9.130:44557/ using document root <none> and password file <none>
I20260809 02:13:47.724476 25499 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:47.724520 25499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:47.724617 25499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:47.725755 25499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "9d846dc480e44935b7ee92d3c41d05f6"
format_stamp: "Formatted at 2026-08-09 02:13:47 on dist-test-slave-p3t5"
server_key: "471ecf09ede4bd11427ad5006428d993"
server_key_iv: "c62b97bf66460638e0adb3c99f4ee66f"
server_key_version: "encryptionkey@0"
I20260809 02:13:47.726109 25499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "9d846dc480e44935b7ee92d3c41d05f6"
format_stamp: "Formatted at 2026-08-09 02:13:47 on dist-test-slave-p3t5"
server_key: "471ecf09ede4bd11427ad5006428d993"
server_key_iv: "c62b97bf66460638e0adb3c99f4ee66f"
server_key_version: "encryptionkey@0"
I20260809 02:13:47.727504 25499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:13:47.728257 25515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:47.728575 25499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:47.728696 25499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "9d846dc480e44935b7ee92d3c41d05f6"
format_stamp: "Formatted at 2026-08-09 02:13:47 on dist-test-slave-p3t5"
server_key: "471ecf09ede4bd11427ad5006428d993"
server_key_iv: "c62b97bf66460638e0adb3c99f4ee66f"
server_key_version: "encryptionkey@0"
I20260809 02:13:47.728782 25499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:47.745175 25499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:47.747663 25499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:47.747861 25499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:47.748131 25499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:47.748497 25499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:47.748556 25499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:47.748606 25499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:47.748636 25499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:47.754849 25499 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:39777
I20260809 02:13:47.754894 25637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:39777 every 8 connection(s)
I20260809 02:13:47.755288 25499 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:13:47.759776 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 25499
I20260809 02:13:47.759905 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:13:47.760103 5158 external_mini_cluster.cc:1428] Setting key 6d34e523c7ce973b6850ff2a4e02f3b9
Aug 09 02:13:47 dist-test-slave-p3t5 krb5kdc[25237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241627, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.130@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:13:47.766676 25328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:47.756155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:50957 (local address 127.5.9.190:40999)
0809 02:13:47.756218 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:47.756220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:47.756427 (+ 207us) server_negotiation.cc:407] Connection header received
0809 02:13:47.756966 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:47.756968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:47.756978 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:47.757044 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:47.758153 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.758661 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.759733 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.760054 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.760131 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:47.763135 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:47.763137 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:47.763137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:47.763145 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:47.764948 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:47.765390 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:47.765392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:47.765393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:47.765470 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:47.765825 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:47.765826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:47.765827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:47.766028 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:47.766113 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:47.766470 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 02:13:47.766578 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 02:13:47.766908 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:47.756123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:40999 (local address 127.5.9.130:50957)
0809 02:13:47.756286 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:47.756331 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:47.756864 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:47.757111 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:47.757115 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:47.757843 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:47.758052 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:47.758057 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.758785 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.758786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:47.759630 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:47.759633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.760196 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.760198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:47.760377 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:47.761088 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:13:47.761100 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:47.763038 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:47.765043 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:47.765050 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:47.765053 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:47.765307 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:47.765536 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:47.765537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:47.765538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:47.765745 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:47.766139 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:47.766145 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:47.766286 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 02:13:47.766426 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 02:13:47.766612 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260809 02:13:47.767509 25638 heartbeater.cc:347] Connected to a master server at 127.5.9.190:40999
I20260809 02:13:47.767640 25638 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:47.767884 25638 heartbeater.cc:511] Master 127.5.9.190:40999 requested a full tablet report, sending...
I20260809 02:13:47.768379 25286 ts_manager.cc:194] Registered new tserver with Master: 9d846dc480e44935b7ee92d3c41d05f6 (127.5.9.130:39777)
I20260809 02:13:47.768786 25286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.130@KRBTEST.COM'} at 127.5.9.130:50957
WARNING: no policy specified for kudu/127.5.9.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
Entry for principal kudu/127.5.9.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab.
I20260809 02:13:47.793197 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:40999
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:40477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:47.878542 25643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:47.878803 25643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:47.878842 25643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:47.880081 25643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:47.880127 25643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:47.880154 25643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:13:47.881738 25643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:40477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.131.keytab
--principal=kudu/127.5.9.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:47.881989 25643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:47.882238 25643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:47.887570 25648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:47.887573 25643 server_base.cc:1034] running on GCE node
W20260809 02:13:47.887557 25649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:47.887550 25651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:47.888099 25643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:47.888319 25643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:47.889494 25643 hybrid_clock.cc:648] HybridClock initialized: now 1786241627889480 us; error 30 us; skew 500 ppm
Aug 09 02:13:47 dist-test-slave-p3t5 krb5kdc[25237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241627, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:47.892206 25643 init.cc:377] Logged in from keytab as kudu/127.5.9.131@KRBTEST.COM (short username kudu)
I20260809 02:13:47.892642 25643 webserver.cc:460] Webserver started at http://127.5.9.131:38013/ using document root <none> and password file <none>
I20260809 02:13:47.892843 25643 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:47.892885 25643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:47.892985 25643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:47.894102 25643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "fa8ca9d5e1854313885188e22faed0c3"
format_stamp: "Formatted at 2026-08-09 02:13:47 on dist-test-slave-p3t5"
server_key: "e1b7ed45ecf74e0373690a525f7ee251"
server_key_iv: "cd0f5668dd374a6d66cdf1c8eb788383"
server_key_version: "encryptionkey@0"
I20260809 02:13:47.894438 25643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "fa8ca9d5e1854313885188e22faed0c3"
format_stamp: "Formatted at 2026-08-09 02:13:47 on dist-test-slave-p3t5"
server_key: "e1b7ed45ecf74e0373690a525f7ee251"
server_key_iv: "cd0f5668dd374a6d66cdf1c8eb788383"
server_key_version: "encryptionkey@0"
I20260809 02:13:47.895778 25643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:13:47.896584 25658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:47.896762 25643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:47.896853 25643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "fa8ca9d5e1854313885188e22faed0c3"
format_stamp: "Formatted at 2026-08-09 02:13:47 on dist-test-slave-p3t5"
server_key: "e1b7ed45ecf74e0373690a525f7ee251"
server_key_iv: "cd0f5668dd374a6d66cdf1c8eb788383"
server_key_version: "encryptionkey@0"
I20260809 02:13:47.896919 25643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:47.926529 25643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:47.928699 25643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:47.928823 25643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:47.929010 25643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:47.929376 25643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:47.929426 25643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:47.929474 25643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:47.929507 25643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:47.935982 25643 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:45929
I20260809 02:13:47.936074 25782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:45929 every 8 connection(s)
I20260809 02:13:47.936381 25643 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:13:47.939914 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 25643
I20260809 02:13:47.940038 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:13:47.940208 5158 external_mini_cluster.cc:1428] Setting key cb9dc76fc6dd6429594320787554c87b
Aug 09 02:13:47 dist-test-slave-p3t5 krb5kdc[25237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241627, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.131@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:13:47.946228 25328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:47.937195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:36855 (local address 127.5.9.190:40999)
0809 02:13:47.937245 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:47.937247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:47.937444 (+ 197us) server_negotiation.cc:407] Connection header received
0809 02:13:47.938034 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:47.938035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:47.938047 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:47.938102 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:47.939086 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.939596 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.940892 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.941157 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.941227 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:47.943498 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:47.943499 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:47.943500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:47.943508 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:47.944727 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:47.945150 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:47.945151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:47.945152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:47.945221 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:47.945488 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:47.945488 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:47.945489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:47.945662 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:47.945729 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:47.946037 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 02:13:47.946129 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:13:47.946429 25786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:47.937166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:40999 (local address 127.5.9.131:36855)
0809 02:13:47.937330 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:47.937368 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:47.937939 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:47.938159 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:47.938163 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:47.938775 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:47.938989 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:47.938994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.939681 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.939682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:47.940765 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:47.940768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.941250 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.941250 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:47.941388 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:47.941923 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:13:47.941932 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:47.943394 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:47.944816 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:47.944824 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:47.944826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:47.945071 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:47.945289 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:13:47.945290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:13:47.945290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:13:47.945408 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:13:47.945719 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:47.945726 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:47.945879 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 02:13:47.946001 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 02:13:47.946177 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260809 02:13:47.946926 25783 heartbeater.cc:347] Connected to a master server at 127.5.9.190:40999
I20260809 02:13:47.947048 25783 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:47.947232 25783 heartbeater.cc:511] Master 127.5.9.190:40999 requested a full tablet report, sending...
I20260809 02:13:47.947623 25286 ts_manager.cc:194] Registered new tserver with Master: fa8ca9d5e1854313885188e22faed0c3 (127.5.9.131:45929)
I20260809 02:13:47.947992 25286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.9.131@KRBTEST.COM'} at 127.5.9.131:36855
I20260809 02:13:47.952145 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:13:47.959894 25328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:47.953583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58350 (local address 127.5.9.190:40999)
0809 02:13:47.953676 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:47.953678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:47.953769 (+ 91us) server_negotiation.cc:407] Connection header received
0809 02:13:47.953895 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:47.953896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:47.953907 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:47.953965 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:47.954547 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.955048 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.955805 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.956011 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.956085 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:47.957366 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:47.957368 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:47.957368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:47.957376 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:47.958666 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:47.959028 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:47.959029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:47.959029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:47.959098 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:47.959301 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:47.959302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:47.959302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:47.959469 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:47.959579 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:47.959698 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 02:13:47.959783 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 02:13:47.961390 25286 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:13:47.963207 25286 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:13:47.970254 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:47.965961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34688 (local address 127.5.9.129:46465)
0809 02:13:47.966150 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:47.966155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:47.966163 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:13:47.966197 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:47.966198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:47.966210 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:47.966298 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:47.966494 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.967590 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.969086 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.969806 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.969880 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:47.969945 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:47.970147 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 02:13:47.970183 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":95,"threads_started":1}
I20260809 02:13:47.970271 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:47.965787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:46465 (local address 127.0.0.1:34688)
0809 02:13:47.965919 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:47.965929 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:47.965982 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:47.966315 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:47.966318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:47.966321 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:47.966397 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:47.966400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.967694 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.967694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:47.968990 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:47.968992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.969911 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.969912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:47.970014 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:47.970030 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 02:13:47.970113 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 02:13:47.970169 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":31,"thread_start_us":79,"threads_started":1}
I20260809 02:13:47.970723 25800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:47.965557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:39777 (local address 127.0.0.1:50568)
0809 02:13:47.965911 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:47.965922 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:47.965982 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:47.966694 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:47.966695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:47.966698 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:47.966766 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:47.966767 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.967985 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.967986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:47.968901 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:47.968904 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.970495 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.970495 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:47.970564 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:47.970567 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:13:47.970611 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 02:13:47.970651 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":21,"thread_start_us":72,"threads_started":1}
I20260809 02:13:47.971017 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:47.965707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50568 (local address 127.5.9.130:39777)
0809 02:13:47.966462 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:47.966466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:47.966477 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:13:47.966517 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:47.966519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:47.966535 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:47.966609 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:47.966864 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.967873 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.968988 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.969803 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.969882 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:47.969947 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:47.970878 (+ 931us) server_negotiation.cc:299] Negotiation successful
0809 02:13:47.970935 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":88,"threads_started":1}
I20260809 02:13:47.971107 25421 tablet_service.cc:1467] Processing CreateTablet for tablet 32aa1dca0c5342e8b656d74daba61886 (DEFAULT_TABLE table=test-table [id=5298a53430624a44acaae2e8c73e6536]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:47.971285 25796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:47.965555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:45929 (local address 127.0.0.1:36096)
0809 02:13:47.965757 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:47.965802 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:47.965873 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:47.966436 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:47.966437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:47.966439 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:47.966501 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:47.966503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.967548 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.967549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:47.968989 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:47.968992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.971036 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.971037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:47.971108 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:47.971112 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:13:47.971167 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 02:13:47.971210 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":47,"thread_start_us":123,"threads_started":1}
I20260809 02:13:47.971520 25798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:47.965672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36096 (local address 127.5.9.131:45929)
0809 02:13:47.966158 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:47.966162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:47.966170 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:13:47.966202 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:47.966203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:47.966215 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:47.966310 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:47.966588 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.967445 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.969709 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:47.970887 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:47.970949 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:47.971000 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:47.971403 (+ 403us) server_negotiation.cc:299] Negotiation successful
0809 02:13:47.971451 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":83,"threads_started":1}
I20260809 02:13:47.971539 25421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32aa1dca0c5342e8b656d74daba61886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:47.971692 25565 tablet_service.cc:1467] Processing CreateTablet for tablet 32aa1dca0c5342e8b656d74daba61886 (DEFAULT_TABLE table=test-table [id=5298a53430624a44acaae2e8c73e6536]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:47.971997 25565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32aa1dca0c5342e8b656d74daba61886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:47.972066 25710 tablet_service.cc:1467] Processing CreateTablet for tablet 32aa1dca0c5342e8b656d74daba61886 (DEFAULT_TABLE table=test-table [id=5298a53430624a44acaae2e8c73e6536]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:47.972298 25710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32aa1dca0c5342e8b656d74daba61886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:47.974979 25802 tablet_bootstrap.cc:492] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3: Bootstrap starting.
I20260809 02:13:47.975374 25803 tablet_bootstrap.cc:492] T 32aa1dca0c5342e8b656d74daba61886 P 83f528b9721b42b58fe95216cd85a36e: Bootstrap starting.
I20260809 02:13:47.975955 25802 tablet_bootstrap.cc:654] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:47.976038 25803 tablet_bootstrap.cc:654] T 32aa1dca0c5342e8b656d74daba61886 P 83f528b9721b42b58fe95216cd85a36e: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:47.976603 25802 log.cc:824] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:47.976632 25803 log.cc:824] T 32aa1dca0c5342e8b656d74daba61886 P 83f528b9721b42b58fe95216cd85a36e: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:47.976960 25804 tablet_bootstrap.cc:492] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6: Bootstrap starting.
I20260809 02:13:47.977663 25804 tablet_bootstrap.cc:654] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:47.977706 25802 tablet_bootstrap.cc:492] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3: No bootstrap required, opened a new log
I20260809 02:13:47.977723 25803 tablet_bootstrap.cc:492] T 32aa1dca0c5342e8b656d74daba61886 P 83f528b9721b42b58fe95216cd85a36e: No bootstrap required, opened a new log
I20260809 02:13:47.977778 25802 ts_tablet_manager.cc:1397] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:13:47.977802 25803 ts_tablet_manager.cc:1397] T 32aa1dca0c5342e8b656d74daba61886 P 83f528b9721b42b58fe95216cd85a36e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:13:47.978272 25804 log.cc:824] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:47.979240 25804 tablet_bootstrap.cc:492] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6: No bootstrap required, opened a new log
I20260809 02:13:47.979316 25804 ts_tablet_manager.cc:1397] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:13:47.979727 25802 raft_consensus.cc:348] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa8ca9d5e1854313885188e22faed0c3" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45929 } } peers { permanent_uuid: "9d846dc480e44935b7ee92d3c41d05f6" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39777 } } peers { permanent_uuid: "83f528b9721b42b58fe95216cd85a36e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46465 } }
I20260809 02:13:47.979903 25802 raft_consensus.cc:374] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:47.979795 25803 raft_consensus.cc:348] T 32aa1dca0c5342e8b656d74daba61886 P 83f528b9721b42b58fe95216cd85a36e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa8ca9d5e1854313885188e22faed0c3" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45929 } } peers { permanent_uuid: "9d846dc480e44935b7ee92d3c41d05f6" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39777 } } peers { permanent_uuid: "83f528b9721b42b58fe95216cd85a36e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46465 } }
I20260809 02:13:47.979946 25802 raft_consensus.cc:729] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa8ca9d5e1854313885188e22faed0c3, State: Initialized, Role: FOLLOWER
I20260809 02:13:47.979965 25803 raft_consensus.cc:374] T 32aa1dca0c5342e8b656d74daba61886 P 83f528b9721b42b58fe95216cd85a36e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:47.980000 25803 raft_consensus.cc:729] T 32aa1dca0c5342e8b656d74daba61886 P 83f528b9721b42b58fe95216cd85a36e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f528b9721b42b58fe95216cd85a36e, State: Initialized, Role: FOLLOWER
I20260809 02:13:47.980058 25802 consensus_queue.cc:260] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa8ca9d5e1854313885188e22faed0c3" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45929 } } peers { permanent_uuid: "9d846dc480e44935b7ee92d3c41d05f6" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39777 } } peers { permanent_uuid: "83f528b9721b42b58fe95216cd85a36e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46465 } }
I20260809 02:13:47.980116 25803 consensus_queue.cc:260] T 32aa1dca0c5342e8b656d74daba61886 P 83f528b9721b42b58fe95216cd85a36e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa8ca9d5e1854313885188e22faed0c3" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45929 } } peers { permanent_uuid: "9d846dc480e44935b7ee92d3c41d05f6" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39777 } } peers { permanent_uuid: "83f528b9721b42b58fe95216cd85a36e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46465 } }
I20260809 02:13:47.980357 25802 ts_tablet_manager.cc:1428] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 02:13:47.980366 25783 heartbeater.cc:503] Master 127.5.9.190:40999 was elected leader, sending a full tablet report...
I20260809 02:13:47.980393 25803 ts_tablet_manager.cc:1428] T 32aa1dca0c5342e8b656d74daba61886 P 83f528b9721b42b58fe95216cd85a36e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:13:47.980518 25493 heartbeater.cc:503] Master 127.5.9.190:40999 was elected leader, sending a full tablet report...
I20260809 02:13:47.981293 25804 raft_consensus.cc:348] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa8ca9d5e1854313885188e22faed0c3" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45929 } } peers { permanent_uuid: "9d846dc480e44935b7ee92d3c41d05f6" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39777 } } peers { permanent_uuid: "83f528b9721b42b58fe95216cd85a36e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46465 } }
I20260809 02:13:47.981490 25804 raft_consensus.cc:374] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:47.981544 25804 raft_consensus.cc:729] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d846dc480e44935b7ee92d3c41d05f6, State: Initialized, Role: FOLLOWER
I20260809 02:13:47.981696 25804 consensus_queue.cc:260] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa8ca9d5e1854313885188e22faed0c3" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45929 } } peers { permanent_uuid: "9d846dc480e44935b7ee92d3c41d05f6" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39777 } } peers { permanent_uuid: "83f528b9721b42b58fe95216cd85a36e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46465 } }
I20260809 02:13:47.982015 25804 ts_tablet_manager.cc:1428] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:13:47.982034 25638 heartbeater.cc:503] Master 127.5.9.190:40999 was elected leader, sending a full tablet report...
W20260809 02:13:48.005987 25639 tablet.cc:2369] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:13:48.076647 25809 raft_consensus.cc:482] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:13:48.076797 25809 raft_consensus.cc:504] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa8ca9d5e1854313885188e22faed0c3" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45929 } } peers { permanent_uuid: "9d846dc480e44935b7ee92d3c41d05f6" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39777 } } peers { permanent_uuid: "83f528b9721b42b58fe95216cd85a36e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46465 } }
I20260809 02:13:48.077129 25809 leader_election.cc:288] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d846dc480e44935b7ee92d3c41d05f6 (127.5.9.130:39777), 83f528b9721b42b58fe95216cd85a36e (127.5.9.129:46465)
I20260809 02:13:48.080149 25786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.077335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:39777 (local address 127.5.9.131:39355)
0809 02:13:48.077411 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:48.077425 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:48.077511 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:48.077703 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:48.077704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:48.077707 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:48.077804 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.077806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.078536 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.078537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.079324 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.079327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.079876 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.079877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.079951 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.079954 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:13:48.080028 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 02:13:48.080069 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 02:13:48.080149 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.077468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:39355 (local address 127.5.9.130:39777)
0809 02:13:48.077542 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:48.077544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:48.077555 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:13:48.077587 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:48.077588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:48.077602 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:48.077681 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:48.077961 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.078449 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.079420 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.079791 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.079869 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.079880 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:48.080036 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 02:13:48.080079 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 02:13:48.080318 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.077602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:47119 (local address 127.5.9.129:46465)
0809 02:13:48.077675 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:48.077678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:48.077709 (+ 31us) server_negotiation.cc:407] Connection header received
0809 02:13:48.077789 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:48.077790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:48.077802 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:48.077869 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:48.078075 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.078607 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.079601 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.079978 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.080060 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.080069 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:48.080215 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 02:13:48.080251 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:13:48.080331 25812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.077459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:46465 (local address 127.5.9.131:47119)
0809 02:13:48.077661 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:48.077670 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:48.077720 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:48.077887 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:48.077888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:48.077890 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:48.077989 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.077991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.078684 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.078685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.079512 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.079514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.080062 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.080063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.080139 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.080143 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:13:48.080207 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 02:13:48.080247 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":107,"threads_started":1}
I20260809 02:13:48.080546 25587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32aa1dca0c5342e8b656d74daba61886" candidate_uuid: "fa8ca9d5e1854313885188e22faed0c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d846dc480e44935b7ee92d3c41d05f6" is_pre_election: true
I20260809 02:13:48.080751 25587 raft_consensus.cc:2454] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa8ca9d5e1854313885188e22faed0c3 in term 0.
I20260809 02:13:48.080900 25443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32aa1dca0c5342e8b656d74daba61886" candidate_uuid: "fa8ca9d5e1854313885188e22faed0c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f528b9721b42b58fe95216cd85a36e" is_pre_election: true
I20260809 02:13:48.081012 25661 leader_election.cc:302] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d846dc480e44935b7ee92d3c41d05f6, fa8ca9d5e1854313885188e22faed0c3; no voters:
I20260809 02:13:48.081108 25443 raft_consensus.cc:2454] T 32aa1dca0c5342e8b656d74daba61886 P 83f528b9721b42b58fe95216cd85a36e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa8ca9d5e1854313885188e22faed0c3 in term 0.
I20260809 02:13:48.081126 25809 raft_consensus.cc:2781] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:13:48.081173 25809 raft_consensus.cc:482] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:13:48.081204 25809 raft_consensus.cc:3037] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:48.082145 25809 raft_consensus.cc:504] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa8ca9d5e1854313885188e22faed0c3" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45929 } } peers { permanent_uuid: "9d846dc480e44935b7ee92d3c41d05f6" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39777 } } peers { permanent_uuid: "83f528b9721b42b58fe95216cd85a36e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46465 } }
I20260809 02:13:48.082283 25809 leader_election.cc:288] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [CANDIDATE]: Term 1 election: Requested vote from peers 9d846dc480e44935b7ee92d3c41d05f6 (127.5.9.130:39777), 83f528b9721b42b58fe95216cd85a36e (127.5.9.129:46465)
I20260809 02:13:48.082481 25587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32aa1dca0c5342e8b656d74daba61886" candidate_uuid: "fa8ca9d5e1854313885188e22faed0c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d846dc480e44935b7ee92d3c41d05f6"
I20260809 02:13:48.082474 25443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32aa1dca0c5342e8b656d74daba61886" candidate_uuid: "fa8ca9d5e1854313885188e22faed0c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f528b9721b42b58fe95216cd85a36e"
I20260809 02:13:48.082561 25443 raft_consensus.cc:3037] T 32aa1dca0c5342e8b656d74daba61886 P 83f528b9721b42b58fe95216cd85a36e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:48.082557 25587 raft_consensus.cc:3037] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:48.083266 25587 raft_consensus.cc:2454] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa8ca9d5e1854313885188e22faed0c3 in term 1.
I20260809 02:13:48.083343 25443 raft_consensus.cc:2454] T 32aa1dca0c5342e8b656d74daba61886 P 83f528b9721b42b58fe95216cd85a36e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa8ca9d5e1854313885188e22faed0c3 in term 1.
I20260809 02:13:48.083462 25661 leader_election.cc:302] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d846dc480e44935b7ee92d3c41d05f6, fa8ca9d5e1854313885188e22faed0c3; no voters:
I20260809 02:13:48.083595 25809 raft_consensus.cc:2781] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:48.083819 25809 raft_consensus.cc:686] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [term 1 LEADER]: Becoming Leader. State: Replica: fa8ca9d5e1854313885188e22faed0c3, State: Running, Role: LEADER
I20260809 02:13:48.083921 25809 consensus_queue.cc:237] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa8ca9d5e1854313885188e22faed0c3" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45929 } } peers { permanent_uuid: "9d846dc480e44935b7ee92d3c41d05f6" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39777 } } peers { permanent_uuid: "83f528b9721b42b58fe95216cd85a36e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46465 } }
I20260809 02:13:48.084733 25286 catalog_manager.cc:5495] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa8ca9d5e1854313885188e22faed0c3 (127.5.9.131). New cstate: current_term: 1 leader_uuid: "fa8ca9d5e1854313885188e22faed0c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa8ca9d5e1854313885188e22faed0c3" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 45929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d846dc480e44935b7ee92d3c41d05f6" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f528b9721b42b58fe95216cd85a36e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46465 } health_report { overall_health: UNKNOWN } } }
I20260809 02:13:48.090901 25798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.088181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36110 (local address 127.5.9.131:45929)
0809 02:13:48.088228 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:48.088230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:48.088243 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:13:48.088349 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:48.088350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:48.088376 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:48.088458 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:13:48.088587 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.089092 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.090038 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.090250 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.090341 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.090528 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:13:48.090634 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:13:48.090724 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:48.090759 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 02:13:48.090780 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 02:13:48.091353 25495 tablet.cc:2369] T 32aa1dca0c5342e8b656d74daba61886 P 83f528b9721b42b58fe95216cd85a36e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:13:48.092694 25443 raft_consensus.cc:1261] T 32aa1dca0c5342e8b656d74daba61886 P 83f528b9721b42b58fe95216cd85a36e [term 1 FOLLOWER]: Refusing update from remote peer fa8ca9d5e1854313885188e22faed0c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:13:48.092725 25587 raft_consensus.cc:1261] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6 [term 1 FOLLOWER]: Refusing update from remote peer fa8ca9d5e1854313885188e22faed0c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:13:48.093084 25809 consensus_queue.cc:1035] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f528b9721b42b58fe95216cd85a36e" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 46465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:13:48.093245 25813 consensus_queue.cc:1035] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d846dc480e44935b7ee92d3c41d05f6" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 39777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:13:48.096001 25817 mvcc.cc:205] Tried to move back new op lower bound from 7316445708665192448 to 7316445708632354816. Current Snapshot: MvccSnapshot[applied={T|T < 7316445708665192448}]
I20260809 02:13:48.096618 25818 mvcc.cc:205] Tried to move back new op lower bound from 7316445708665192448 to 7316445708632354816. Current Snapshot: MvccSnapshot[applied={T|T < 7316445708665192448}]
I20260809 02:13:48.098171 25285 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:13:48.098284 25285 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:13:48.099061 25285 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6d2ede22e0b47c09169d184badff524: Sending DeleteTablet for 3 replicas of tablet 32aa1dca0c5342e8b656d74daba61886
I20260809 02:13:48.099357 25710 tablet_service.cc:1514] Processing DeleteTablet for tablet 32aa1dca0c5342e8b656d74daba61886 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:48 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:36096
I20260809 02:13:48.099529 25421 tablet_service.cc:1514] Processing DeleteTablet for tablet 32aa1dca0c5342e8b656d74daba61886 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:48 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:34688
I20260809 02:13:48.099732 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 25351
I20260809 02:13:48.099807 25565 tablet_service.cc:1514] Processing DeleteTablet for tablet 32aa1dca0c5342e8b656d74daba61886 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:48 UTC) from {username='kudu', principal='kudu/127.5.9.190@KRBTEST.COM'} at 127.0.0.1:50568
I20260809 02:13:48.101379 25828 tablet_replica.cc:330] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6: stopping tablet replica
I20260809 02:13:48.101514 25828 raft_consensus.cc:2229] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:13:48.101660 25828 raft_consensus.cc:2258] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:13:48.101840 25826 tablet_replica.cc:330] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3: stopping tablet replica
I20260809 02:13:48.101940 25826 raft_consensus.cc:2229] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:13:48.102037 25826 raft_consensus.cc:2258] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:13:48.102778 25826 ts_tablet_manager.cc:1905] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:13:48.103020 25828 ts_tablet_manager.cc:1905] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:13:48.104488 25826 ts_tablet_manager.cc:1918] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:13:48.104554 25826 log.cc:1196] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/32aa1dca0c5342e8b656d74daba61886
I20260809 02:13:48.104790 25826 ts_tablet_manager.cc:1939] T 32aa1dca0c5342e8b656d74daba61886 P fa8ca9d5e1854313885188e22faed0c3: Deleting consensus metadata
I20260809 02:13:48.105223 25273 catalog_manager.cc:4841] TS fa8ca9d5e1854313885188e22faed0c3 (127.5.9.131:45929): tablet 32aa1dca0c5342e8b656d74daba61886 (table test-table [id=5298a53430624a44acaae2e8c73e6536]) successfully deleted
I20260809 02:13:48.105602 25828 ts_tablet_manager.cc:1918] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:13:48.105799 25828 log.cc:1196] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/32aa1dca0c5342e8b656d74daba61886
I20260809 02:13:48.106204 25828 ts_tablet_manager.cc:1939] T 32aa1dca0c5342e8b656d74daba61886 P 9d846dc480e44935b7ee92d3c41d05f6: Deleting consensus metadata
I20260809 02:13:48.106825 25271 catalog_manager.cc:4841] TS 9d846dc480e44935b7ee92d3c41d05f6 (127.5.9.130:39777): tablet 32aa1dca0c5342e8b656d74daba61886 (table test-table [id=5298a53430624a44acaae2e8c73e6536]) successfully deleted
W20260809 02:13:48.108680 25271 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:46465 (error 108)
I20260809 02:13:48.109236 25796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.109113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:46465 (local address 127.0.0.1:34692)
0809 02:13:48.109165 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:48.109202 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:46465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 02:13:48.109247 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 25499
I20260809 02:13:48.117974 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 25643
W20260809 02:13:48.117974 25271 catalog_manager.cc:4584] TS 83f528b9721b42b58fe95216cd85a36e (127.5.9.129:46465): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 32aa1dca0c5342e8b656d74daba61886: Network error: Client connection negotiation failed: client connection to 127.5.9.129:46465: connect: Connection refused (error 111)
I20260809 02:13:48.124498 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 25251
2026-08-09T02:13:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2065 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T02:13:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:13:48Z Disabled control of system clock
I20260809 02:13:48.157936 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:32819
--webserver_interface=127.5.9.190
--webserver_port=0
--builtin_ntp_servers=127.5.9.148:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:32819
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:13:48.248675 25833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:48.248919 25833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:48.248939 25833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:48.250052 25833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:13:48.250089 25833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:48.250102 25833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:48.250115 25833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:13:48.250128 25833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:13:48.251416 25833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:32819
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:32819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:48.251621 25833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:48.251873 25833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:48.256876 25838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:48.256843 25841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:48.256896 25833 server_base.cc:1034] running on GCE node
W20260809 02:13:48.256865 25839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:48.257325 25833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:48.257686 25833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:48.258878 25833 hybrid_clock.cc:648] HybridClock initialized: now 1786241628258871 us; error 34 us; skew 500 ppm
I20260809 02:13:48.260205 25833 webserver.cc:460] Webserver started at http://127.5.9.190:43445/ using document root <none> and password file <none>
I20260809 02:13:48.260473 25833 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:48.260552 25833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:48.260692 25833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:48.261950 25833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "d2bbf82b83004d539bec58bca5c92a03"
format_stamp: "Formatted at 2026-08-09 02:13:48 on dist-test-slave-p3t5"
server_key: "b39d81a8a61f7e89c432b2f101512239"
server_key_iv: "d217c4a2d5d602e0859b2e70847467d5"
server_key_version: "encryptionkey@0"
I20260809 02:13:48.262367 25833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "d2bbf82b83004d539bec58bca5c92a03"
format_stamp: "Formatted at 2026-08-09 02:13:48 on dist-test-slave-p3t5"
server_key: "b39d81a8a61f7e89c432b2f101512239"
server_key_iv: "d217c4a2d5d602e0859b2e70847467d5"
server_key_version: "encryptionkey@0"
I20260809 02:13:48.263761 25833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 02:13:48.264591 25847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:48.264794 25833 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:48.264901 25833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "d2bbf82b83004d539bec58bca5c92a03"
format_stamp: "Formatted at 2026-08-09 02:13:48 on dist-test-slave-p3t5"
server_key: "b39d81a8a61f7e89c432b2f101512239"
server_key_iv: "d217c4a2d5d602e0859b2e70847467d5"
server_key_version: "encryptionkey@0"
I20260809 02:13:48.264988 25833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:48.278855 25833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:48.279229 25833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:48.279397 25833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:48.283960 25833 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:32819
I20260809 02:13:48.284005 25906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:32819 every 8 connection(s)
I20260809 02:13:48.284396 25833 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:13:48.285470 25907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:48.288043 25907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03: Bootstrap starting.
I20260809 02:13:48.288733 25907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:48.289083 25907 log.cc:824] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:48.289963 25907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03: No bootstrap required, opened a new log
I20260809 02:13:48.291697 25907 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bbf82b83004d539bec58bca5c92a03" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 32819 } }
I20260809 02:13:48.291860 25907 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:48.291884 25907 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2bbf82b83004d539bec58bca5c92a03, State: Initialized, Role: FOLLOWER
I20260809 02:13:48.292052 25907 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bbf82b83004d539bec58bca5c92a03" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 32819 } }
I20260809 02:13:48.292130 25907 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:13:48.292155 25907 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:13:48.292204 25907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:48.292809 25907 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bbf82b83004d539bec58bca5c92a03" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 32819 } }
I20260809 02:13:48.292930 25907 leader_election.cc:302] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2bbf82b83004d539bec58bca5c92a03; no voters:
I20260809 02:13:48.293161 25907 leader_election.cc:288] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:13:48.293282 25910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:48.293476 25907 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:13:48.293745 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 25833
I20260809 02:13:48.293881 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:13:48.294093 5158 external_mini_cluster.cc:1428] Setting key 99b7ab828c3554a3ee1898db2b7b0813
I20260809 02:13:48.294297 25910 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [term 1 LEADER]: Becoming Leader. State: Replica: d2bbf82b83004d539bec58bca5c92a03, State: Running, Role: LEADER
I20260809 02:13:48.294428 25910 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bbf82b83004d539bec58bca5c92a03" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 32819 } }
I20260809 02:13:48.295068 25911 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2bbf82b83004d539bec58bca5c92a03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bbf82b83004d539bec58bca5c92a03" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 32819 } } }
I20260809 02:13:48.295159 25911 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:48.295277 25912 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2bbf82b83004d539bec58bca5c92a03. Latest consensus state: current_term: 1 leader_uuid: "d2bbf82b83004d539bec58bca5c92a03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bbf82b83004d539bec58bca5c92a03" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 32819 } } }
I20260809 02:13:48.295338 25912 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:48.295584 25927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:13:48.296218 25927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:13:48.298463 25927 catalog_manager.cc:1347] Generated new cluster ID: a2d8d5b596e8445993af2782e3bb8cd5
I20260809 02:13:48.298527 25927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:13:48.300632 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.294866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42092 (local address 127.5.9.190:32819)
0809 02:13:48.295104 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:48.295113 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:48.295300 (+ 187us) server_negotiation.cc:407] Connection header received
0809 02:13:48.295612 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:48.295621 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:48.295838 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:48.296010 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:48.296250 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.297450 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.298393 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.299003 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.299123 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.299883 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:48.299887 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:48.299888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:48.300109 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:48.300156 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:48.300170 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:48.300235 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:48.300331 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 02:13:48.300414 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":74,"threads_started":1}
I20260809 02:13:48.304266 25927 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:13:48.304795 25927 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:13:48.310441 25927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03: Generated new TSK 0
I20260809 02:13:48.310671 25927 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:13:48.315239 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.129:0
--local_ip_for_outbound_sockets=127.5.9.129
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:32819
--builtin_ntp_servers=127.5.9.148:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:13:48.402709 25932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:48.402983 25932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:48.403025 25932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:48.404335 25932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:48.404398 25932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:48.404453 25932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.129
I20260809 02:13:48.406078 25932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.9.129
--webserver_port=0
--tserver_master_addrs=127.5.9.190:32819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.129
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:48.406400 25932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:48.406772 25932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:48.412400 25937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:48.412611 25932 server_base.cc:1034] running on GCE node
W20260809 02:13:48.412431 25938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:48.412431 25940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:48.413003 25932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:48.413314 25932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:48.414496 25932 hybrid_clock.cc:648] HybridClock initialized: now 1786241628414456 us; error 64 us; skew 500 ppm
I20260809 02:13:48.416030 25932 webserver.cc:460] Webserver started at http://127.5.9.129:43679/ using document root <none> and password file <none>
I20260809 02:13:48.416313 25932 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:48.416375 25932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:48.416543 25932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:48.417755 25932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/data/instance:
uuid: "c3cbc98723df463ea0c48aad28456791"
format_stamp: "Formatted at 2026-08-09 02:13:48 on dist-test-slave-p3t5"
server_key: "542c4c7078b8b4d9ebf24866896ecae0"
server_key_iv: "6e045d562d10b1b1293a7cf5b1cee110"
server_key_version: "encryptionkey@0"
I20260809 02:13:48.418145 25932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance:
uuid: "c3cbc98723df463ea0c48aad28456791"
format_stamp: "Formatted at 2026-08-09 02:13:48 on dist-test-slave-p3t5"
server_key: "542c4c7078b8b4d9ebf24866896ecae0"
server_key_iv: "6e045d562d10b1b1293a7cf5b1cee110"
server_key_version: "encryptionkey@0"
I20260809 02:13:48.419569 25932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:13:48.420382 25946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:48.420555 25932 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:13:48.420672 25932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/wal
uuid: "c3cbc98723df463ea0c48aad28456791"
format_stamp: "Formatted at 2026-08-09 02:13:48 on dist-test-slave-p3t5"
server_key: "542c4c7078b8b4d9ebf24866896ecae0"
server_key_iv: "6e045d562d10b1b1293a7cf5b1cee110"
server_key_version: "encryptionkey@0"
I20260809 02:13:48.420754 25932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:48.441903 25932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:48.442273 25932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:48.442443 25932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:48.442713 25932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:48.443092 25932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:48.443138 25932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:48.443162 25932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:48.443176 25932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:48.449748 25932 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.129:45381
I20260809 02:13:48.449800 26064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.129:45381 every 8 connection(s)
I20260809 02:13:48.450186 25932 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/data/info.pb
I20260809 02:13:48.451937 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 25932
I20260809 02:13:48.452059 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-0/wal/instance
I20260809 02:13:48.452273 5158 external_mini_cluster.cc:1428] Setting key 7e06665a52929ef3c1d8624ca344e0ca
I20260809 02:13:48.453262 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.130:0
--local_ip_for_outbound_sockets=127.5.9.130
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:32819
--builtin_ntp_servers=127.5.9.148:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:13:48.456306 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.451019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.129:53183 (local address 127.5.9.190:32819)
0809 02:13:48.451083 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:48.451085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:48.451365 (+ 280us) server_negotiation.cc:407] Connection header received
0809 02:13:48.451937 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:48.451939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:48.451956 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:48.452016 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:48.452885 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.453671 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.454684 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.454909 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.454997 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.455984 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:48.455986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:48.455986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:48.455991 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:48.456008 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:48.456013 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:48.456084 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:48.456185 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 02:13:48.456221 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 02:13:48.456570 26068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.451000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:32819 (local address 127.5.9.129:53183)
0809 02:13:48.451234 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:48.451271 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:48.451812 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:48.452136 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:48.452141 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:48.452156 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:48.452780 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.452786 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.453760 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.453761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.454580 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.454583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.455012 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.455012 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.455156 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.455809 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:13:48.455809 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:48.455852 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:13:48.455885 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:48.456087 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:48.456097 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:48.456100 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:13:48.456189 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 02:13:48.456308 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":110,"threads_started":1}
I20260809 02:13:48.457113 26066 heartbeater.cc:347] Connected to a master server at 127.5.9.190:32819
I20260809 02:13:48.457219 26066 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:48.457419 26066 heartbeater.cc:511] Master 127.5.9.190:32819 requested a full tablet report, sending...
I20260809 02:13:48.457989 25867 ts_manager.cc:194] Registered new tserver with Master: c3cbc98723df463ea0c48aad28456791 (127.5.9.129:45381)
I20260809 02:13:48.458582 25867 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.9.129:53183
W20260809 02:13:48.539515 26069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:48.539768 26069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:48.539806 26069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:48.540977 26069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:48.541039 26069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:48.541082 26069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.130
I20260809 02:13:48.542574 26069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.9.130
--webserver_port=0
--tserver_master_addrs=127.5.9.190:32819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.130
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:48.542855 26069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:48.543164 26069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:48.548339 26076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:48.548447 26069 server_base.cc:1034] running on GCE node
W20260809 02:13:48.548342 26078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:48.548368 26074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:48.548856 26069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:48.549144 26069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:48.550321 26069 hybrid_clock.cc:648] HybridClock initialized: now 1786241628550307 us; error 46 us; skew 500 ppm
I20260809 02:13:48.551720 26069 webserver.cc:460] Webserver started at http://127.5.9.130:41901/ using document root <none> and password file <none>
I20260809 02:13:48.552012 26069 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:48.552078 26069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:48.552235 26069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:48.553382 26069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/data/instance:
uuid: "b2537dbb88ec439487d122713ae497cf"
format_stamp: "Formatted at 2026-08-09 02:13:48 on dist-test-slave-p3t5"
server_key: "167b5995de549fbfd2a0f94033918736"
server_key_iv: "3dbf7e69be86146d743575e81f69c64c"
server_key_version: "encryptionkey@0"
I20260809 02:13:48.553800 26069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance:
uuid: "b2537dbb88ec439487d122713ae497cf"
format_stamp: "Formatted at 2026-08-09 02:13:48 on dist-test-slave-p3t5"
server_key: "167b5995de549fbfd2a0f94033918736"
server_key_iv: "3dbf7e69be86146d743575e81f69c64c"
server_key_version: "encryptionkey@0"
I20260809 02:13:48.555171 26069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:13:48.555959 26084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:48.556134 26069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:13:48.556238 26069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/wal
uuid: "b2537dbb88ec439487d122713ae497cf"
format_stamp: "Formatted at 2026-08-09 02:13:48 on dist-test-slave-p3t5"
server_key: "167b5995de549fbfd2a0f94033918736"
server_key_iv: "3dbf7e69be86146d743575e81f69c64c"
server_key_version: "encryptionkey@0"
I20260809 02:13:48.556313 26069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:48.583191 26069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:48.583564 26069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:48.583731 26069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:48.584022 26069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:48.584398 26069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:48.584455 26069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:48.584493 26069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:48.584537 26069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:48.590587 26069 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.130:43077
I20260809 02:13:48.590637 26202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.130:43077 every 8 connection(s)
I20260809 02:13:48.590991 26069 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/data/info.pb
I20260809 02:13:48.596330 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.591741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.130:53825 (local address 127.5.9.190:32819)
0809 02:13:48.591811 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:48.591812 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:48.592062 (+ 250us) server_negotiation.cc:407] Connection header received
0809 02:13:48.592572 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:48.592574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:48.592586 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:48.592641 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:48.593428 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.594061 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.595084 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.595269 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.595328 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.596114 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:48.596116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:48.596116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:48.596120 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:48.596135 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:48.596139 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:48.596186 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:48.596243 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 02:13:48.596269 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:13:48.596583 26205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.591724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:32819 (local address 127.5.9.130:53825)
0809 02:13:48.591936 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:48.591972 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:48.592482 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:48.592722 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:48.592726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:48.592735 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:48.593350 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.593356 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.594160 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.594161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.594960 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.594962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.595357 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.595358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.595474 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.595985 (+ 511us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:13:48.595985 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:48.596014 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:13:48.596038 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:48.596189 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:48.596192 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:48.596194 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:13:48.596240 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 02:13:48.596321 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260809 02:13:48.597126 26203 heartbeater.cc:347] Connected to a master server at 127.5.9.190:32819
I20260809 02:13:48.597224 26203 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:48.597379 26203 heartbeater.cc:511] Master 127.5.9.190:32819 requested a full tablet report, sending...
I20260809 02:13:48.597752 25867 ts_manager.cc:194] Registered new tserver with Master: b2537dbb88ec439487d122713ae497cf (127.5.9.130:43077)
I20260809 02:13:48.598099 25867 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.9.130:53825
I20260809 02:13:48.600142 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 26069
I20260809 02:13:48.600245 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/wal/instance
I20260809 02:13:48.600416 5158 external_mini_cluster.cc:1428] Setting key 3c5173bff47eb595f88ad36a19bbad1c
I20260809 02:13:48.601323 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.9.131:0
--local_ip_for_outbound_sockets=127.5.9.131
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:32819
--builtin_ntp_servers=127.5.9.148:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:13:48.686017 26206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:48.686246 26206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:48.686268 26206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:48.687510 26206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:48.687557 26206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:48.687583 26206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.9.131
I20260809 02:13:48.688925 26206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.9.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.9.131
--webserver_port=0
--tserver_master_addrs=127.5.9.190:32819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.9.131
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:48.689128 26206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:48.689400 26206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:48.694686 26211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:48.694686 26214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:48.694707 26212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:48.695021 26206 server_base.cc:1034] running on GCE node
I20260809 02:13:48.695302 26206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:48.695578 26206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:48.696733 26206 hybrid_clock.cc:648] HybridClock initialized: now 1786241628696728 us; error 42 us; skew 500 ppm
I20260809 02:13:48.698143 26206 webserver.cc:460] Webserver started at http://127.5.9.131:43351/ using document root <none> and password file <none>
I20260809 02:13:48.698416 26206 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:48.698487 26206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:48.698637 26206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:48.699790 26206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/data/instance:
uuid: "efb28fa02dbc4024a819e1a5d91ab753"
format_stamp: "Formatted at 2026-08-09 02:13:48 on dist-test-slave-p3t5"
server_key: "f176acdbe816f60ee3849ea1bea8f8cb"
server_key_iv: "c50a4360c4bc129448e1f4ed9f481f19"
server_key_version: "encryptionkey@0"
I20260809 02:13:48.700156 26206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance:
uuid: "efb28fa02dbc4024a819e1a5d91ab753"
format_stamp: "Formatted at 2026-08-09 02:13:48 on dist-test-slave-p3t5"
server_key: "f176acdbe816f60ee3849ea1bea8f8cb"
server_key_iv: "c50a4360c4bc129448e1f4ed9f481f19"
server_key_version: "encryptionkey@0"
I20260809 02:13:48.701473 26206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:13:48.702279 26221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:48.702454 26206 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:13:48.702554 26206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/wal
uuid: "efb28fa02dbc4024a819e1a5d91ab753"
format_stamp: "Formatted at 2026-08-09 02:13:48 on dist-test-slave-p3t5"
server_key: "f176acdbe816f60ee3849ea1bea8f8cb"
server_key_iv: "c50a4360c4bc129448e1f4ed9f481f19"
server_key_version: "encryptionkey@0"
I20260809 02:13:48.702634 26206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:48.718555 26206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:48.718899 26206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:48.719060 26206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:48.719318 26206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:13:48.719667 26206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:13:48.719719 26206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:48.719774 26206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:13:48.719805 26206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:48.725823 26206 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.131:35539
I20260809 02:13:48.725878 26342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.131:35539 every 8 connection(s)
I20260809 02:13:48.726251 26206 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/data/info.pb
I20260809 02:13:48.727589 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 26206
I20260809 02:13:48.727690 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/wal/instance
I20260809 02:13:48.727947 5158 external_mini_cluster.cc:1428] Setting key db5c86f1c23cdc24c9aeb48b9482d2e1
I20260809 02:13:48.731801 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.727029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:52021 (local address 127.5.9.190:32819)
0809 02:13:48.727077 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:48.727079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:48.727318 (+ 239us) server_negotiation.cc:407] Connection header received
0809 02:13:48.727936 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:48.727938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:48.727954 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:48.728031 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:48.728922 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.729473 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.730533 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.730717 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.730770 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.731588 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:48.731589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:48.731589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:48.731594 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:48.731609 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:48.731613 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:48.731656 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:48.731709 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 02:13:48.731737 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:13:48.732067 26345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.727018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:32819 (local address 127.5.9.131:52021)
0809 02:13:48.727185 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:48.727221 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:48.727804 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:48.728122 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:48.728126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:48.728138 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:13:48.728824 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.728832 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.729576 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.729578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.730449 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.730452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.730789 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.730790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.730934 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.731457 (+ 523us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:13:48.731457 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:13:48.731487 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:13:48.731512 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:13:48.731656 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:13:48.731659 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:13:48.731661 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:13:48.731704 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 02:13:48.731784 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260809 02:13:48.732577 26343 heartbeater.cc:347] Connected to a master server at 127.5.9.190:32819
I20260809 02:13:48.732694 26343 heartbeater.cc:464] Registering TS with master...
I20260809 02:13:48.732896 26343 heartbeater.cc:511] Master 127.5.9.190:32819 requested a full tablet report, sending...
I20260809 02:13:48.733299 25867 ts_manager.cc:194] Registered new tserver with Master: efb28fa02dbc4024a819e1a5d91ab753 (127.5.9.131:35539)
I20260809 02:13:48.733755 25867 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.9.131:52021
I20260809 02:13:48.740051 5158 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:13:48.744395 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.741518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42098 (local address 127.5.9.190:32819)
0809 02:13:48.741589 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:48.741590 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:48.741767 (+ 177us) server_negotiation.cc:407] Connection header received
0809 02:13:48.741832 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:48.741833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:48.741845 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:48.741905 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:48.742050 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.742534 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.743317 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.743501 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.743555 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.744187 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:48.744188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:48.744188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:48.744193 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:48.744206 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:48.744210 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:48.744254 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:48.744303 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 02:13:48.744328 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 02:13:48.746064 25867 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:13:48.747828 25867 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:13:48.754534 26353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.750133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:45381 (local address 127.0.0.1:44636)
0809 02:13:48.750873 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:48.750883 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:48.750925 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:48.751189 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:48.751190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:48.751192 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:48.751233 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.751235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.752490 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.752491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.753466 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.753470 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.754206 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.754208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.754292 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.754307 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 02:13:48.754383 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:13:48.754447 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":13,"thread_start_us":88,"threads_started":1}
I20260809 02:13:48.754701 26351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.750116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:43077 (local address 127.0.0.1:42946)
0809 02:13:48.750311 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:48.750353 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:48.750422 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:48.750605 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:48.750607 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:48.750609 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:48.750678 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.750680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.752299 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.752300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.753364 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.753367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.754466 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.754467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.754530 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.754533 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:13:48.754582 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 02:13:48.754626 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":3,"thread_start_us":81,"threads_started":1}
I20260809 02:13:48.754738 26356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.750391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44636 (local address 127.5.9.129:45381)
0809 02:13:48.750631 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:48.750635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:48.750988 (+ 353us) server_negotiation.cc:407] Connection header received
0809 02:13:48.751034 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:48.751036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:48.751050 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:48.751121 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:48.751322 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.752410 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.753537 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.754105 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.754193 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.754298 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:48.754399 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 02:13:48.754440 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":95,"threads_started":1}
I20260809 02:13:48.755007 26352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.750132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.131:35539 (local address 127.0.0.1:54130)
0809 02:13:48.750691 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:48.750701 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:48.750784 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:48.750967 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:48.750969 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:48.750971 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:48.751017 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.751019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.752036 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.752037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.753364 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.753367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.754599 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.754600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.754837 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.754842 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:13:48.754905 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 02:13:48.754941 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":4,"thread_start_us":105,"threads_started":1}
I20260809 02:13:48.755003 26354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.750205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42946 (local address 127.5.9.130:43077)
0809 02:13:48.750415 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:48.750420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:48.750432 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:13:48.750500 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:48.750503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:48.750518 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:48.750591 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:48.750782 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.752210 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.753472 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.754260 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.754337 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.754424 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:48.754860 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 02:13:48.754919 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":82,"threads_started":1}
I20260809 02:13:48.755081 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.750367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54130 (local address 127.5.9.131:35539)
0809 02:13:48.750639 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:48.750643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:48.750773 (+ 130us) server_negotiation.cc:407] Connection header received
0809 02:13:48.750857 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:48.750859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:48.750875 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:48.750969 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:48.751082 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.751863 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.753695 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.754418 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.754485 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.754567 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:48.754946 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 02:13:48.754993 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":96,"threads_started":1}
I20260809 02:13:48.755519 25996 tablet_service.cc:1467] Processing CreateTablet for tablet 582768fa241c47a0bf252805f15ef8b3 (DEFAULT_TABLE table=test-table [id=c121c8410ab5451d927d7250b2495c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:48.755720 26134 tablet_service.cc:1467] Processing CreateTablet for tablet 582768fa241c47a0bf252805f15ef8b3 (DEFAULT_TABLE table=test-table [id=c121c8410ab5451d927d7250b2495c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:48.755712 26273 tablet_service.cc:1467] Processing CreateTablet for tablet 582768fa241c47a0bf252805f15ef8b3 (DEFAULT_TABLE table=test-table [id=c121c8410ab5451d927d7250b2495c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:13:48.755859 25996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 582768fa241c47a0bf252805f15ef8b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:48.756048 26273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 582768fa241c47a0bf252805f15ef8b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:48.756048 26134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 582768fa241c47a0bf252805f15ef8b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:48.758654 26357 tablet_bootstrap.cc:492] T 582768fa241c47a0bf252805f15ef8b3 P c3cbc98723df463ea0c48aad28456791: Bootstrap starting.
I20260809 02:13:48.759423 26357 tablet_bootstrap.cc:654] T 582768fa241c47a0bf252805f15ef8b3 P c3cbc98723df463ea0c48aad28456791: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:48.759780 26357 log.cc:824] T 582768fa241c47a0bf252805f15ef8b3 P c3cbc98723df463ea0c48aad28456791: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:48.759919 26359 tablet_bootstrap.cc:492] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf: Bootstrap starting.
I20260809 02:13:48.760638 26357 tablet_bootstrap.cc:492] T 582768fa241c47a0bf252805f15ef8b3 P c3cbc98723df463ea0c48aad28456791: No bootstrap required, opened a new log
I20260809 02:13:48.760651 26359 tablet_bootstrap.cc:654] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:48.760725 26357 ts_tablet_manager.cc:1397] T 582768fa241c47a0bf252805f15ef8b3 P c3cbc98723df463ea0c48aad28456791: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:13:48.760982 26359 log.cc:824] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:48.761303 26358 tablet_bootstrap.cc:492] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753: Bootstrap starting.
I20260809 02:13:48.761976 26359 tablet_bootstrap.cc:492] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf: No bootstrap required, opened a new log
I20260809 02:13:48.761989 26358 tablet_bootstrap.cc:654] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:48.762058 26359 ts_tablet_manager.cc:1397] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:13:48.762300 26358 log.cc:824] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:48.763255 26358 tablet_bootstrap.cc:492] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753: No bootstrap required, opened a new log
I20260809 02:13:48.763320 26358 ts_tablet_manager.cc:1397] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 02:13:48.763343 26357 raft_consensus.cc:348] T 582768fa241c47a0bf252805f15ef8b3 P c3cbc98723df463ea0c48aad28456791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2537dbb88ec439487d122713ae497cf" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43077 } } peers { permanent_uuid: "efb28fa02dbc4024a819e1a5d91ab753" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "c3cbc98723df463ea0c48aad28456791" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45381 } }
I20260809 02:13:48.763516 26357 raft_consensus.cc:374] T 582768fa241c47a0bf252805f15ef8b3 P c3cbc98723df463ea0c48aad28456791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:48.763566 26357 raft_consensus.cc:729] T 582768fa241c47a0bf252805f15ef8b3 P c3cbc98723df463ea0c48aad28456791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3cbc98723df463ea0c48aad28456791, State: Initialized, Role: FOLLOWER
I20260809 02:13:48.763705 26357 consensus_queue.cc:260] T 582768fa241c47a0bf252805f15ef8b3 P c3cbc98723df463ea0c48aad28456791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2537dbb88ec439487d122713ae497cf" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43077 } } peers { permanent_uuid: "efb28fa02dbc4024a819e1a5d91ab753" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "c3cbc98723df463ea0c48aad28456791" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45381 } }
I20260809 02:13:48.764016 26066 heartbeater.cc:503] Master 127.5.9.190:32819 was elected leader, sending a full tablet report...
I20260809 02:13:48.764009 26357 ts_tablet_manager.cc:1428] T 582768fa241c47a0bf252805f15ef8b3 P c3cbc98723df463ea0c48aad28456791: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 02:13:48.764743 26359 raft_consensus.cc:348] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2537dbb88ec439487d122713ae497cf" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43077 } } peers { permanent_uuid: "efb28fa02dbc4024a819e1a5d91ab753" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "c3cbc98723df463ea0c48aad28456791" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45381 } }
I20260809 02:13:48.764928 26359 raft_consensus.cc:374] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:48.764976 26359 raft_consensus.cc:729] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2537dbb88ec439487d122713ae497cf, State: Initialized, Role: FOLLOWER
I20260809 02:13:48.765092 26359 consensus_queue.cc:260] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2537dbb88ec439487d122713ae497cf" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43077 } } peers { permanent_uuid: "efb28fa02dbc4024a819e1a5d91ab753" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "c3cbc98723df463ea0c48aad28456791" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45381 } }
I20260809 02:13:48.765353 26359 ts_tablet_manager.cc:1428] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260809 02:13:48.765451 26203 heartbeater.cc:503] Master 127.5.9.190:32819 was elected leader, sending a full tablet report...
I20260809 02:13:48.766021 26358 raft_consensus.cc:348] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2537dbb88ec439487d122713ae497cf" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43077 } } peers { permanent_uuid: "efb28fa02dbc4024a819e1a5d91ab753" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "c3cbc98723df463ea0c48aad28456791" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45381 } }
I20260809 02:13:48.766140 26358 raft_consensus.cc:374] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:48.766181 26358 raft_consensus.cc:729] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efb28fa02dbc4024a819e1a5d91ab753, State: Initialized, Role: FOLLOWER
I20260809 02:13:48.766290 26358 consensus_queue.cc:260] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2537dbb88ec439487d122713ae497cf" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43077 } } peers { permanent_uuid: "efb28fa02dbc4024a819e1a5d91ab753" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "c3cbc98723df463ea0c48aad28456791" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45381 } }
I20260809 02:13:48.766548 26358 ts_tablet_manager.cc:1428] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 02:13:48.766600 26343 heartbeater.cc:503] Master 127.5.9.190:32819 was elected leader, sending a full tablet report...
W20260809 02:13:48.841564 26204 tablet.cc:2369] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:13:48.841794 26365 raft_consensus.cc:482] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:13:48.841950 26365 raft_consensus.cc:504] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2537dbb88ec439487d122713ae497cf" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43077 } } peers { permanent_uuid: "efb28fa02dbc4024a819e1a5d91ab753" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "c3cbc98723df463ea0c48aad28456791" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45381 } }
I20260809 02:13:48.842365 26365 leader_election.cc:288] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2537dbb88ec439487d122713ae497cf (127.5.9.130:43077), c3cbc98723df463ea0c48aad28456791 (127.5.9.129:45381)
I20260809 02:13:48.845536 26356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.842721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:57419 (local address 127.5.9.129:45381)
0809 02:13:48.842779 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:48.842781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:48.842794 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:13:48.842819 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:48.842820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:48.842830 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:48.842895 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:48.843059 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.843691 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.844665 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.845175 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.845232 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.845241 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:48.845417 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 02:13:48.845454 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 02:13:48.845553 26345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.842576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:45381 (local address 127.5.9.131:57419)
0809 02:13:48.842627 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:48.842641 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:48.842700 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:48.842899 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:48.842900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:48.842902 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:48.842977 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.842979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.843784 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.843786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.844564 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.844566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.845260 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.845261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.845333 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.845338 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:13:48.845401 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 02:13:48.845453 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 02:13:48.845854 26366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.842578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.130:43077 (local address 127.5.9.131:50385)
0809 02:13:48.842871 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:48.842884 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:13:48.842974 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:13:48.843134 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:13:48.843135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:13:48.843138 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:13:48.843210 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.843212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.844044 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.844045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.844906 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:13:48.844909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.845561 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.845562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:13:48.845661 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.845666 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:13:48.845723 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 02:13:48.845765 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":93,"thread_start_us":129,"threads_started":1}
I20260809 02:13:48.845896 26354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.842636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.9.131:50385 (local address 127.5.9.130:43077)
0809 02:13:48.842697 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:48.842699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:48.842966 (+ 267us) server_negotiation.cc:407] Connection header received
0809 02:13:48.843033 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:48.843034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:48.843047 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:48.843114 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:13:48.843287 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.843954 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.844998 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.845473 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.845539 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.845548 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:48.845747 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 02:13:48.845796 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 02:13:48.846177 26017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "582768fa241c47a0bf252805f15ef8b3" candidate_uuid: "efb28fa02dbc4024a819e1a5d91ab753" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3cbc98723df463ea0c48aad28456791" is_pre_election: true
I20260809 02:13:48.846200 26155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "582768fa241c47a0bf252805f15ef8b3" candidate_uuid: "efb28fa02dbc4024a819e1a5d91ab753" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2537dbb88ec439487d122713ae497cf" is_pre_election: true
I20260809 02:13:48.846364 26017 raft_consensus.cc:2454] T 582768fa241c47a0bf252805f15ef8b3 P c3cbc98723df463ea0c48aad28456791 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efb28fa02dbc4024a819e1a5d91ab753 in term 0.
I20260809 02:13:48.846374 26155 raft_consensus.cc:2454] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efb28fa02dbc4024a819e1a5d91ab753 in term 0.
I20260809 02:13:48.846611 26224 leader_election.cc:302] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2537dbb88ec439487d122713ae497cf, efb28fa02dbc4024a819e1a5d91ab753; no voters:
I20260809 02:13:48.846771 26365 raft_consensus.cc:2781] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:13:48.846835 26365 raft_consensus.cc:482] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:13:48.846853 26365 raft_consensus.cc:3037] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:48.847592 26365 raft_consensus.cc:504] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2537dbb88ec439487d122713ae497cf" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43077 } } peers { permanent_uuid: "efb28fa02dbc4024a819e1a5d91ab753" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "c3cbc98723df463ea0c48aad28456791" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45381 } }
I20260809 02:13:48.847754 26365 leader_election.cc:288] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [CANDIDATE]: Term 1 election: Requested vote from peers b2537dbb88ec439487d122713ae497cf (127.5.9.130:43077), c3cbc98723df463ea0c48aad28456791 (127.5.9.129:45381)
I20260809 02:13:48.847945 26155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "582768fa241c47a0bf252805f15ef8b3" candidate_uuid: "efb28fa02dbc4024a819e1a5d91ab753" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2537dbb88ec439487d122713ae497cf"
I20260809 02:13:48.847950 26017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "582768fa241c47a0bf252805f15ef8b3" candidate_uuid: "efb28fa02dbc4024a819e1a5d91ab753" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3cbc98723df463ea0c48aad28456791"
I20260809 02:13:48.848032 26155 raft_consensus.cc:3037] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:48.848032 26017 raft_consensus.cc:3037] T 582768fa241c47a0bf252805f15ef8b3 P c3cbc98723df463ea0c48aad28456791 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:48.848999 26155 raft_consensus.cc:2454] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efb28fa02dbc4024a819e1a5d91ab753 in term 1.
I20260809 02:13:48.848999 26017 raft_consensus.cc:2454] T 582768fa241c47a0bf252805f15ef8b3 P c3cbc98723df463ea0c48aad28456791 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efb28fa02dbc4024a819e1a5d91ab753 in term 1.
I20260809 02:13:48.849197 26225 leader_election.cc:302] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3cbc98723df463ea0c48aad28456791, efb28fa02dbc4024a819e1a5d91ab753; no voters:
I20260809 02:13:48.849303 26365 raft_consensus.cc:2781] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:48.849447 26365 raft_consensus.cc:686] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [term 1 LEADER]: Becoming Leader. State: Replica: efb28fa02dbc4024a819e1a5d91ab753, State: Running, Role: LEADER
I20260809 02:13:48.849702 26365 consensus_queue.cc:237] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2537dbb88ec439487d122713ae497cf" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43077 } } peers { permanent_uuid: "efb28fa02dbc4024a819e1a5d91ab753" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } } peers { permanent_uuid: "c3cbc98723df463ea0c48aad28456791" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45381 } }
I20260809 02:13:48.850608 25867 catalog_manager.cc:5495] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 reported cstate change: term changed from 0 to 1, leader changed from <none> to efb28fa02dbc4024a819e1a5d91ab753 (127.5.9.131). New cstate: current_term: 1 leader_uuid: "efb28fa02dbc4024a819e1a5d91ab753" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2537dbb88ec439487d122713ae497cf" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efb28fa02dbc4024a819e1a5d91ab753" member_type: VOTER last_known_addr { host: "127.5.9.131" port: 35539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3cbc98723df463ea0c48aad28456791" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45381 } health_report { overall_health: UNKNOWN } } }
I20260809 02:13:48.871692 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.869278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54134 (local address 127.5.9.131:35539)
0809 02:13:48.869341 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:48.869348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:48.869355 (+ 7us) server_negotiation.cc:407] Connection header received
0809 02:13:48.869392 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:48.869393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:48.869409 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:48.869488 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:13:48.869669 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.870140 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.870892 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:48.871091 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:48.871163 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:48.871338 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:13:48.871444 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:13:48.871536 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:48.871560 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 02:13:48.871579 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:13:48.873729 26155 raft_consensus.cc:1261] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf [term 1 FOLLOWER]: Refusing update from remote peer efb28fa02dbc4024a819e1a5d91ab753: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:13:48.873778 26017 raft_consensus.cc:1261] T 582768fa241c47a0bf252805f15ef8b3 P c3cbc98723df463ea0c48aad28456791 [term 1 FOLLOWER]: Refusing update from remote peer efb28fa02dbc4024a819e1a5d91ab753: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:13:48.874118 26367 consensus_queue.cc:1035] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2537dbb88ec439487d122713ae497cf" member_type: VOTER last_known_addr { host: "127.5.9.130" port: 43077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:13:48.874245 26365 consensus_queue.cc:1035] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3cbc98723df463ea0c48aad28456791" member_type: VOTER last_known_addr { host: "127.5.9.129" port: 45381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:13:48.877311 26373 mvcc.cc:205] Tried to move back new op lower bound from 7316445711863959552 to 7316445711769260032. Current Snapshot: MvccSnapshot[applied={T|T < 7316445711863959552}]
I20260809 02:13:48.878791 25866 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:13:48.878886 25866 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:13:48.879681 25866 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d2bbf82b83004d539bec58bca5c92a03: Sending DeleteTablet for 3 replicas of tablet 582768fa241c47a0bf252805f15ef8b3
I20260809 02:13:48.880079 25996 tablet_service.cc:1514] Processing DeleteTablet for tablet 582768fa241c47a0bf252805f15ef8b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:48 UTC) from {username='slave'} at 127.0.0.1:44636
I20260809 02:13:48.880100 26273 tablet_service.cc:1514] Processing DeleteTablet for tablet 582768fa241c47a0bf252805f15ef8b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:48 UTC) from {username='slave'} at 127.0.0.1:54130
I20260809 02:13:48.880292 26134 tablet_service.cc:1514] Processing DeleteTablet for tablet 582768fa241c47a0bf252805f15ef8b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:13:48 UTC) from {username='slave'} at 127.0.0.1:42946
I20260809 02:13:48.880487 26382 tablet_replica.cc:330] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753: stopping tablet replica
I20260809 02:13:48.880594 26382 raft_consensus.cc:2229] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:13:48.880695 26382 raft_consensus.cc:2258] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:13:48.880898 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 25932
I20260809 02:13:48.883838 26382 ts_tablet_manager.cc:1905] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:13:48.887753 26382 ts_tablet_manager.cc:1918] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:13:48.887890 26382 log.cc:1196] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-2/wal/wals/582768fa241c47a0bf252805f15ef8b3
I20260809 02:13:48.888289 26382 ts_tablet_manager.cc:1939] T 582768fa241c47a0bf252805f15ef8b3 P efb28fa02dbc4024a819e1a5d91ab753: Deleting consensus metadata
I20260809 02:13:48.888887 25848 catalog_manager.cc:4841] TS efb28fa02dbc4024a819e1a5d91ab753 (127.5.9.131:35539): tablet 582768fa241c47a0bf252805f15ef8b3 (table test-table [id=c121c8410ab5451d927d7250b2495c03]) successfully deleted
I20260809 02:13:48.889165 26383 tablet_replica.cc:330] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf: stopping tablet replica
I20260809 02:13:48.889325 26383 raft_consensus.cc:2229] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:13:48.889492 26383 raft_consensus.cc:2258] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:13:48.890390 26383 ts_tablet_manager.cc:1905] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:13:48.892607 26383 ts_tablet_manager.cc:1918] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:13:48.892694 26383 log.cc:1196] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf: Deleting WAL directory at /tmp/dist-test-tasktks196/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241377393328-5158-0/minicluster-data/ts-1/wal/wals/582768fa241c47a0bf252805f15ef8b3
I20260809 02:13:48.893013 26383 ts_tablet_manager.cc:1939] T 582768fa241c47a0bf252805f15ef8b3 P b2537dbb88ec439487d122713ae497cf: Deleting consensus metadata
I20260809 02:13:48.893491 25851 catalog_manager.cc:4841] TS b2537dbb88ec439487d122713ae497cf (127.5.9.130:43077): tablet 582768fa241c47a0bf252805f15ef8b3 (table test-table [id=c121c8410ab5451d927d7250b2495c03]) successfully deleted
W20260809 02:13:48.898315 25852 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.9.129:45381 (error 108)
I20260809 02:13:48.898532 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 26069
I20260809 02:13:48.902923 26353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:48.902727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.129:45381 (local address 127.0.0.1:44648)
0809 02:13:48.902804 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 02:13:48.902860 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.129:45381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 02:13:48.903084 25852 catalog_manager.cc:4584] TS c3cbc98723df463ea0c48aad28456791 (127.5.9.129:45381): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 582768fa241c47a0bf252805f15ef8b3: Network error: Client connection negotiation failed: client connection to 127.5.9.129:45381: connect: Connection refused (error 111)
I20260809 02:13:48.907775 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 26206
I20260809 02:13:48.913264 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 25833
2026-08-09T02:13:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (786 ms)
[----------] 6 tests from EncryptionPolicyTest (9372 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:13:48 dist-test-slave-p3t5 krb5kdc[26387](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:13:48 dist-test-slave-p3t5 krb5kdc[26387](info): set up 2 sockets
Aug 09 02:13:48 dist-test-slave-p3t5 krb5kdc[26387](info): commencing operation
krb5kdc: starting...
W20260809 02:13:49.985942 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:13:50 dist-test-slave-p3t5 krb5kdc[26387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:13:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:13:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:13:50.116623 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42729
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:50.202617 26401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:50.202850 26401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:50.202878 26401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:50.204016 26401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:13:50.204056 26401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:50.204069 26401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:50.204082 26401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:13:50.204094 26401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:13:50.205566 26401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42729
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:50.205817 26401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:50.206007 26401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:50.211161 26407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:50.211127 26411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:50.211406 26408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:50.211431 26401 server_base.cc:1034] running on GCE node
I20260809 02:13:50.211714 26401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:50.212031 26401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:50.213178 26401 hybrid_clock.cc:648] HybridClock initialized: now 1786241630213162 us; error 30 us; skew 500 ppm
Aug 09 02:13:50 dist-test-slave-p3t5 krb5kdc[26387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241630, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:50.222868 26401 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:13:50.223440 26401 webserver.cc:460] Webserver started at http://127.0.0.1:43301/ using document root <none> and password file <none>
I20260809 02:13:50.223690 26401 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:50.223738 26401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:50.223903 26401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:50.225050 26401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "c8223212ef2e490abd3be0be68d2b07c"
format_stamp: "Formatted at 2026-08-09 02:13:50 on dist-test-slave-p3t5"
server_key: "9eb3733833d23174b23f37e76e1bf37a"
server_key_iv: "e0e197b9db0c543680a2221ed07c8ced"
server_key_version: "encryptionkey@0"
I20260809 02:13:50.225432 26401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "c8223212ef2e490abd3be0be68d2b07c"
format_stamp: "Formatted at 2026-08-09 02:13:50 on dist-test-slave-p3t5"
server_key: "9eb3733833d23174b23f37e76e1bf37a"
server_key_iv: "e0e197b9db0c543680a2221ed07c8ced"
server_key_version: "encryptionkey@0"
I20260809 02:13:50.226991 26401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:50.227806 26418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:50.228024 26401 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:50.228134 26401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "c8223212ef2e490abd3be0be68d2b07c"
format_stamp: "Formatted at 2026-08-09 02:13:50 on dist-test-slave-p3t5"
server_key: "9eb3733833d23174b23f37e76e1bf37a"
server_key_iv: "e0e197b9db0c543680a2221ed07c8ced"
server_key_version: "encryptionkey@0"
I20260809 02:13:50.228221 26401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:50.247349 26401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:50.249369 26401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:50.249548 26401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:50.253827 26401 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42729
I20260809 02:13:50.253886 26473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42729 every 8 connection(s)
I20260809 02:13:50.254267 26401 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:13:50.255326 26474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:50.257941 26474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c: Bootstrap starting.
I20260809 02:13:50.258538 26474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:50.258812 26474 log.cc:824] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:50.259552 26474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c: No bootstrap required, opened a new log
I20260809 02:13:50.261327 26474 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8223212ef2e490abd3be0be68d2b07c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42729 } }
I20260809 02:13:50.261485 26474 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:50.261509 26474 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8223212ef2e490abd3be0be68d2b07c, State: Initialized, Role: FOLLOWER
I20260809 02:13:50.261699 26474 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8223212ef2e490abd3be0be68d2b07c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42729 } }
I20260809 02:13:50.261782 26474 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:13:50.261806 26474 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:13:50.261838 26474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:50.262461 26474 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8223212ef2e490abd3be0be68d2b07c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42729 } }
I20260809 02:13:50.262508 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 26401
I20260809 02:13:50.262614 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:13:50.262598 26474 leader_election.cc:302] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8223212ef2e490abd3be0be68d2b07c; no voters:
I20260809 02:13:50.262816 26474 leader_election.cc:288] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:13:50.262882 5158 external_mini_cluster.cc:1428] Setting key b499591219f81b5e98151dcd4431d950
I20260809 02:13:50.262878 26477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:50.263049 26477 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [term 1 LEADER]: Becoming Leader. State: Replica: c8223212ef2e490abd3be0be68d2b07c, State: Running, Role: LEADER
I20260809 02:13:50.263136 26474 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:13:50.263141 26477 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8223212ef2e490abd3be0be68d2b07c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42729 } }
I20260809 02:13:50.263664 26477 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8223212ef2e490abd3be0be68d2b07c. Latest consensus state: current_term: 1 leader_uuid: "c8223212ef2e490abd3be0be68d2b07c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8223212ef2e490abd3be0be68d2b07c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42729 } } }
I20260809 02:13:50.263758 26477 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:50.263998 26486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:13:50.264137 26478 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8223212ef2e490abd3be0be68d2b07c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8223212ef2e490abd3be0be68d2b07c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42729 } } }
I20260809 02:13:50.264276 26478 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:50.264650 26486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:13:50.266539 26486 catalog_manager.cc:1347] Generated new cluster ID: 44ce90496258448897e12c42fbb640b6
I20260809 02:13:50.266599 26486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:13:50 dist-test-slave-p3t5 krb5kdc[26387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 02:13:50.276734 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:50.263654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52101 (local address 127.0.0.1:42729)
0809 02:13:50.264005 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:50.264015 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:50.264056 (+ 41us) server_negotiation.cc:407] Connection header received
0809 02:13:50.264189 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:50.264197 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:50.264270 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:50.264436 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:50.265267 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:50.266791 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:50.267982 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:50.268652 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:50.268773 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:50.271868 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:50.271872 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:50.271873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:50.271886 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:50.274804 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:50.275324 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:50.275327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:50.275328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:50.275451 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:50.275724 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:50.275725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:50.275725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:50.276074 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:50.276269 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:50.276358 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 02:13:50.276468 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":83,"threads_started":1}
I20260809 02:13:50.277921 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:13:50.284140 26486 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:13:50.285060 26486 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:13:50.287604 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:50.279549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45911 (local address 127.0.0.1:42729)
0809 02:13:50.279589 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:50.279591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:50.279705 (+ 114us) server_negotiation.cc:407] Connection header received
0809 02:13:50.279759 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:50.279760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:50.279771 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:50.279831 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:50.280493 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:50.281313 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:50.282395 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:50.282733 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:50.282809 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:50.284398 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:50.284399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:50.284400 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:50.284408 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:50.286239 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:50.286733 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:50.286734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:50.286734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:50.286801 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:50.287055 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:50.287056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:50.287057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:50.287187 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:50.287306 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:50.287440 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 02:13:50.287506 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 02:13:50.290091 26486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8223212ef2e490abd3be0be68d2b07c: Generated new TSK 0
I20260809 02:13:50.290295 26486 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:13:50.292778 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 26401
W20260809 02:13:50.299575 26385 connection.cc:535] client connection to 127.0.0.1:42729 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42729): Connection reset by peer (error 104)
2026-08-09T02:13:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1379 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:13:50 dist-test-slave-p3t5 krb5kdc[26509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:13:50 dist-test-slave-p3t5 krb5kdc[26509](info): set up 2 sockets
Aug 09 02:13:50 dist-test-slave-p3t5 krb5kdc[26509](info): commencing operation
krb5kdc: starting...
W20260809 02:13:51.366377 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:13:51 dist-test-slave-p3t5 krb5kdc[26509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:13:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:13:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:13:51.502931 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:51.590471 26524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:51.590713 26524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:51.590740 26524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:51.591970 26524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:13:51.592018 26524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:51.592033 26524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:51.592046 26524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:13:51.592082 26524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:13:51.593652 26524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43853
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:51.593930 26524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:51.594170 26524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:51.599436 26529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:51.599380 26532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:51.599378 26530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:51.599409 26524 server_base.cc:1034] running on GCE node
I20260809 02:13:51.599938 26524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:51.600224 26524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:51.601413 26524 hybrid_clock.cc:648] HybridClock initialized: now 1786241631601395 us; error 38 us; skew 500 ppm
Aug 09 02:13:51 dist-test-slave-p3t5 krb5kdc[26509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241631, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:51.604164 26524 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:13:51.604673 26524 webserver.cc:460] Webserver started at http://127.0.0.1:40849/ using document root <none> and password file <none>
I20260809 02:13:51.604933 26524 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:51.604988 26524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:51.605159 26524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:51.606350 26524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "c46ab4e958a848c285fbfebcc8533944"
format_stamp: "Formatted at 2026-08-09 02:13:51 on dist-test-slave-p3t5"
server_key: "07daf7f9ea6d7c286b568f079914bd57"
server_key_iv: "a522e89eb3d79331f64355576b1afce1"
server_key_version: "encryptionkey@0"
I20260809 02:13:51.606750 26524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "c46ab4e958a848c285fbfebcc8533944"
format_stamp: "Formatted at 2026-08-09 02:13:51 on dist-test-slave-p3t5"
server_key: "07daf7f9ea6d7c286b568f079914bd57"
server_key_iv: "a522e89eb3d79331f64355576b1afce1"
server_key_version: "encryptionkey@0"
I20260809 02:13:51.608217 26524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:13:51.609025 26540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:51.609216 26524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:51.609318 26524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "c46ab4e958a848c285fbfebcc8533944"
format_stamp: "Formatted at 2026-08-09 02:13:51 on dist-test-slave-p3t5"
server_key: "07daf7f9ea6d7c286b568f079914bd57"
server_key_iv: "a522e89eb3d79331f64355576b1afce1"
server_key_version: "encryptionkey@0"
I20260809 02:13:51.609395 26524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:51.621191 26524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:51.623453 26524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:51.623634 26524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:51.628049 26524 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43853
I20260809 02:13:51.628093 26594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43853 every 8 connection(s)
I20260809 02:13:51.628489 26524 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:13:51.629282 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 26524
I20260809 02:13:51.629410 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:13:51.629567 26595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:51.629678 5158 external_mini_cluster.cc:1428] Setting key 2df0ddd3c0475602417ca52db33e977d
I20260809 02:13:51.633154 26595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944: Bootstrap starting.
I20260809 02:13:51.633903 26595 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:51.634289 26595 log.cc:824] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:51.635205 26595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944: No bootstrap required, opened a new log
I20260809 02:13:51.635609 26598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:51.630248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38443 (local address 127.0.0.1:43853)
0809 02:13:51.630427 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:51.630435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:51.630462 (+ 27us) server_negotiation.cc:407] Connection header received
0809 02:13:51.630883 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:51.630890 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:51.630940 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:51.631071 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:51.631325 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:51.632230 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:51.633361 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:51.633880 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:51.633984 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:51.634813 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:51.634818 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:51.634818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:51.635023 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:51.635058 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:51.635070 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:51.635143 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:51.635320 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 02:13:51.635369 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260809 02:13:51.636755 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:13:51.637828 26595 raft_consensus.cc:348] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46ab4e958a848c285fbfebcc8533944" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } }
I20260809 02:13:51.638042 26595 raft_consensus.cc:374] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:51.638108 26595 raft_consensus.cc:729] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c46ab4e958a848c285fbfebcc8533944, State: Initialized, Role: FOLLOWER
I20260809 02:13:51.638247 26595 consensus_queue.cc:260] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46ab4e958a848c285fbfebcc8533944" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } }
I20260809 02:13:51.638362 26595 raft_consensus.cc:388] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:13:51.638434 26595 raft_consensus.cc:482] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:13:51.638496 26595 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:51.639286 26595 raft_consensus.cc:504] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46ab4e958a848c285fbfebcc8533944" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } }
I20260809 02:13:51.639384 26595 leader_election.cc:302] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c46ab4e958a848c285fbfebcc8533944; no voters:
I20260809 02:13:51.639590 26595 leader_election.cc:288] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:13:51.639647 26605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:51.639806 26605 raft_consensus.cc:686] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [term 1 LEADER]: Becoming Leader. State: Replica: c46ab4e958a848c285fbfebcc8533944, State: Running, Role: LEADER
I20260809 02:13:51.639858 26595 sys_catalog.cc:564] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:13:51.640555 26605 consensus_queue.cc:237] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46ab4e958a848c285fbfebcc8533944" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } }
I20260809 02:13:51.641546 26606 sys_catalog.cc:455] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c46ab4e958a848c285fbfebcc8533944. Latest consensus state: current_term: 1 leader_uuid: "c46ab4e958a848c285fbfebcc8533944" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46ab4e958a848c285fbfebcc8533944" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } } }
I20260809 02:13:51.641729 26607 sys_catalog.cc:455] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c46ab4e958a848c285fbfebcc8533944" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46ab4e958a848c285fbfebcc8533944" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } } }
I20260809 02:13:51.641855 26606 sys_catalog.cc:458] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:51.641861 26607 sys_catalog.cc:458] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:51.642122 26598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:51.638477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40157 (local address 127.0.0.1:43853)
0809 02:13:51.638517 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:51.638519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:51.638641 (+ 122us) server_negotiation.cc:407] Connection header received
0809 02:13:51.638717 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:51.638718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:51.638726 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:51.638767 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:51.638931 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:51.639428 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:51.640410 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:51.640742 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:51.640819 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:51.641659 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:51.641660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:51.641660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:51.641665 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:51.641681 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:51.641685 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:51.641741 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:51.641999 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 02:13:51.642011 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 02:13:51.642066 26623 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:13:51.642378 26623 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:13:51.642470 26626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:13:51.642710 26626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:13:51.644304 26626 catalog_manager.cc:1347] Generated new cluster ID: 5a0a4e252fa84a06ac7b80a4a7fa8440
I20260809 02:13:51.644351 26626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:13:51.651427 26626 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:13:51.651993 26626 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:13:51.657261 26626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c46ab4e958a848c285fbfebcc8533944: Generated new TSK 0
I20260809 02:13:51.657500 26626 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:13:51.663697 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 26524
W20260809 02:13:51.669528 26507 connection.cc:535] client connection to 127.0.0.1:43853 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43853): Connection reset by peer (error 104)
2026-08-09T02:13:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1369 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:13:51 dist-test-slave-p3t5 krb5kdc[26630](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:13:51 dist-test-slave-p3t5 krb5kdc[26630](info): set up 2 sockets
Aug 09 02:13:51 dist-test-slave-p3t5 krb5kdc[26630](info): commencing operation
krb5kdc: starting...
W20260809 02:13:52.769472 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:13:52 dist-test-slave-p3t5 krb5kdc[26630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:13:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:13:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:13:52.904819 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43813
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:52.991647 26644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:52.991915 26644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:52.991959 26644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:52.993147 26644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:13:52.993218 26644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:52.993242 26644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:52.993265 26644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:13:52.993297 26644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:13:52.994866 26644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43813
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:52.995168 26644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:52.995412 26644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:53.000676 26649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:53.000663 26650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:53.000663 26652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:53.000695 26644 server_base.cc:1034] running on GCE node
I20260809 02:13:53.001152 26644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:53.001412 26644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:53.002576 26644 hybrid_clock.cc:648] HybridClock initialized: now 1786241633002545 us; error 46 us; skew 500 ppm
Aug 09 02:13:52 dist-test-slave-p3t5 krb5kdc[26630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241632, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:53.005121 26644 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:13:53.005605 26644 webserver.cc:460] Webserver started at http://127.0.0.1:38995/ using document root <none> and password file <none>
I20260809 02:13:53.005903 26644 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:53.005970 26644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:53.006315 26644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:53.007472 26644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "d3e1e20b85284973a2c3ccadfcd752d5"
format_stamp: "Formatted at 2026-08-09 02:13:53 on dist-test-slave-p3t5"
server_key: "6693c9b88fc235e78091aae57f82fc9c"
server_key_iv: "81f829441e1b5c6e1799213a2d3b1aad"
server_key_version: "encryptionkey@0"
I20260809 02:13:53.007865 26644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "d3e1e20b85284973a2c3ccadfcd752d5"
format_stamp: "Formatted at 2026-08-09 02:13:53 on dist-test-slave-p3t5"
server_key: "6693c9b88fc235e78091aae57f82fc9c"
server_key_iv: "81f829441e1b5c6e1799213a2d3b1aad"
server_key_version: "encryptionkey@0"
I20260809 02:13:53.009313 26644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:13:53.010140 26659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:53.010319 26644 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:53.010424 26644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "d3e1e20b85284973a2c3ccadfcd752d5"
format_stamp: "Formatted at 2026-08-09 02:13:53 on dist-test-slave-p3t5"
server_key: "6693c9b88fc235e78091aae57f82fc9c"
server_key_iv: "81f829441e1b5c6e1799213a2d3b1aad"
server_key_version: "encryptionkey@0"
I20260809 02:13:53.010505 26644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:53.013936 26644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:53.016048 26644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:53.016216 26644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:53.020726 26644 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43813
I20260809 02:13:53.020778 26714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43813 every 8 connection(s)
I20260809 02:13:53.021159 26644 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:13:53.022332 26715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:53.024714 26715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5: Bootstrap starting.
I20260809 02:13:53.025300 26715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:53.025555 26715 log.cc:824] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:53.026324 26715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5: No bootstrap required, opened a new log
I20260809 02:13:53.028005 26715 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3e1e20b85284973a2c3ccadfcd752d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } }
I20260809 02:13:53.028149 26715 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:53.028172 26715 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e1e20b85284973a2c3ccadfcd752d5, State: Initialized, Role: FOLLOWER
I20260809 02:13:53.028312 26715 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3e1e20b85284973a2c3ccadfcd752d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } }
I20260809 02:13:53.028399 26715 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:13:53.028422 26715 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:13:53.028451 26715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:53.028975 26715 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3e1e20b85284973a2c3ccadfcd752d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } }
I20260809 02:13:53.029062 26715 leader_election.cc:302] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3e1e20b85284973a2c3ccadfcd752d5; no voters:
I20260809 02:13:53.029189 26715 leader_election.cc:288] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:13:53.029258 26718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:53.029424 26718 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [term 1 LEADER]: Becoming Leader. State: Replica: d3e1e20b85284973a2c3ccadfcd752d5, State: Running, Role: LEADER
I20260809 02:13:53.029485 26715 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:13:53.029547 26718 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3e1e20b85284973a2c3ccadfcd752d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } }
I20260809 02:13:53.029943 26720 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3e1e20b85284973a2c3ccadfcd752d5. Latest consensus state: current_term: 1 leader_uuid: "d3e1e20b85284973a2c3ccadfcd752d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3e1e20b85284973a2c3ccadfcd752d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } } }
I20260809 02:13:53.030102 26719 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3e1e20b85284973a2c3ccadfcd752d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3e1e20b85284973a2c3ccadfcd752d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43813 } } }
I20260809 02:13:53.030184 26719 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:53.030102 26720 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:53.030558 26727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:13:53.030910 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 26644
I20260809 02:13:53.031032 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:13:53.031091 26727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:13:53.031256 5158 external_mini_cluster.cc:1428] Setting key 4cb9e392a5e81fcdaabb80cf55a8d6b6
I20260809 02:13:53.033299 26727 catalog_manager.cc:1347] Generated new cluster ID: 04f0ed0d5dc042c9ad39c019481c9019
I20260809 02:13:53.033360 26727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:13:53.033762 26738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:53.031935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45735 (local address 127.0.0.1:43813)
0809 02:13:53.032100 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:53.032108 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:53.032133 (+ 25us) server_negotiation.cc:407] Connection header received
0809 02:13:53.032257 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:53.032264 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:53.032297 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:53.032465 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:53.033149 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:53.033152 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:53.033152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:53.033156 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:53.033215 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:53.033232 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:53.033336 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:53.033419 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 02:13:53.033464 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260809 02:13:53.034632 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:13:53.043040 26738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:53.036030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49463 (local address 127.0.0.1:43813)
0809 02:13:53.036068 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:53.036070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:53.041904 (+ 5834us) server_negotiation.cc:407] Connection header received
0809 02:13:53.042003 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:53.042005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:53.042011 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:53.042081 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:53.042754 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:53.042755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:53.042756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:53.042760 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:53.042778 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:53.042782 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:53.042854 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:53.042915 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 02:13:53.042932 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:13:53.075682 26727 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:13:53.075933 26727 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:13:53.084280 26727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3e1e20b85284973a2c3ccadfcd752d5: Generated new TSK 0
I20260809 02:13:53.084506 26727 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:13:53.088207 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 26644
2026-08-09T02:13:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:13:53 dist-test-slave-p3t5 krb5kdc[26748](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:13:53 dist-test-slave-p3t5 krb5kdc[26748](info): set up 2 sockets
Aug 09 02:13:53 dist-test-slave-p3t5 krb5kdc[26748](info): commencing operation
krb5kdc: starting...
W20260809 02:13:54.169973 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:13:54 dist-test-slave-p3t5 krb5kdc[26748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:13:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:13:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:13:54.304062 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45295
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:54.388681 26762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:54.388964 26762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:54.389005 26762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:54.390205 26762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:13:54.390285 26762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:54.390314 26762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:54.390345 26762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:13:54.390368 26762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:13:54.391891 26762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:54.392180 26762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:54.392414 26762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:13:54.398031 26762 server_base.cc:1034] running on GCE node
W20260809 02:13:54.398069 26769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:54.398062 26771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:54.398188 26768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:54.398489 26762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:54.398751 26762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:54.399894 26762 hybrid_clock.cc:648] HybridClock initialized: now 1786241634399870 us; error 39 us; skew 500 ppm
Aug 09 02:13:54 dist-test-slave-p3t5 krb5kdc[26748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241634, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:54.402561 26762 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:13:54.403053 26762 webserver.cc:460] Webserver started at http://127.0.0.1:42667/ using document root <none> and password file <none>
I20260809 02:13:54.403292 26762 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:54.403343 26762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:54.403496 26762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:54.404596 26762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "7aae81c4bb854a9b999f49ff2ab52d5d"
format_stamp: "Formatted at 2026-08-09 02:13:54 on dist-test-slave-p3t5"
server_key: "6a104585088f0e7b0c59542e2423377c"
server_key_iv: "37dd100455430099dc4cd5e8abb36b1b"
server_key_version: "encryptionkey@0"
I20260809 02:13:54.404968 26762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "7aae81c4bb854a9b999f49ff2ab52d5d"
format_stamp: "Formatted at 2026-08-09 02:13:54 on dist-test-slave-p3t5"
server_key: "6a104585088f0e7b0c59542e2423377c"
server_key_iv: "37dd100455430099dc4cd5e8abb36b1b"
server_key_version: "encryptionkey@0"
I20260809 02:13:54.406500 26762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:13:54.407303 26778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:54.407508 26762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:54.407609 26762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "7aae81c4bb854a9b999f49ff2ab52d5d"
format_stamp: "Formatted at 2026-08-09 02:13:54 on dist-test-slave-p3t5"
server_key: "6a104585088f0e7b0c59542e2423377c"
server_key_iv: "37dd100455430099dc4cd5e8abb36b1b"
server_key_version: "encryptionkey@0"
I20260809 02:13:54.407671 26762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:54.421203 26762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:54.423579 26762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:54.423748 26762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:54.427928 26762 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45295
I20260809 02:13:54.427978 26832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45295 every 8 connection(s)
I20260809 02:13:54.428372 26762 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:13:54.429481 26833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:54.429993 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 26762
I20260809 02:13:54.430089 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:13:54.430286 5158 external_mini_cluster.cc:1428] Setting key 403a6faf22a5245126737e040e091d56
I20260809 02:13:54.432719 26833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d: Bootstrap starting.
I20260809 02:13:54.433435 26833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:54.433797 26833 log.cc:824] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:54.434578 26833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d: No bootstrap required, opened a new log
I20260809 02:13:54.436462 26833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aae81c4bb854a9b999f49ff2ab52d5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45295 } }
I20260809 02:13:54.436606 26833 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:54.436630 26833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aae81c4bb854a9b999f49ff2ab52d5d, State: Initialized, Role: FOLLOWER
I20260809 02:13:54.436775 26833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aae81c4bb854a9b999f49ff2ab52d5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45295 } }
I20260809 02:13:54.436842 26833 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:13:54.436882 26833 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:13:54.436932 26833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [term 0 FOLLOWER]: Advancing to term 1
Aug 09 02:13:54 dist-test-slave-p3t5 krb5kdc[26748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 02:13:54.437537 26833 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aae81c4bb854a9b999f49ff2ab52d5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45295 } }
I20260809 02:13:54.437681 26833 leader_election.cc:302] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7aae81c4bb854a9b999f49ff2ab52d5d; no voters:
I20260809 02:13:54.437894 26833 leader_election.cc:288] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:13:54.437945 26838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:54.438138 26838 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [term 1 LEADER]: Becoming Leader. State: Replica: 7aae81c4bb854a9b999f49ff2ab52d5d, State: Running, Role: LEADER
I20260809 02:13:54.438187 26833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:13:54.438241 26838 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aae81c4bb854a9b999f49ff2ab52d5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45295 } }
I20260809 02:13:54.438886 26841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7aae81c4bb854a9b999f49ff2ab52d5d. Latest consensus state: current_term: 1 leader_uuid: "7aae81c4bb854a9b999f49ff2ab52d5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aae81c4bb854a9b999f49ff2ab52d5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45295 } } }
I20260809 02:13:54.439003 26841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:54.438886 26839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7aae81c4bb854a9b999f49ff2ab52d5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aae81c4bb854a9b999f49ff2ab52d5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45295 } } }
I20260809 02:13:54.439168 26839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:54.439891 26848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:13:54.440518 26848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:13:54.442028 26836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:54.430883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39463 (local address 127.0.0.1:45295)
0809 02:13:54.431109 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:54.431120 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:54.431158 (+ 38us) server_negotiation.cc:407] Connection header received
0809 02:13:54.431721 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:54.431730 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:54.431794 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:54.431957 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:54.432587 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:54.434048 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:54.434913 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:54.435327 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:54.435453 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:54.438071 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:54.438076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:54.438076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:54.438089 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:54.440196 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:54.440684 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:54.440687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:54.440687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:54.440779 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:54.440992 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:54.440993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:54.440993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:54.441313 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:54.441466 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:54.441663 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 02:13:54.441828 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":103,"threads_started":1}
I20260809 02:13:54.442262 26848 catalog_manager.cc:1347] Generated new cluster ID: 46d0e0096f9a45929544192992d6c286
I20260809 02:13:54.442312 26848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:13:54.443030 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:13:54.451083 26836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:54.444388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48349 (local address 127.0.0.1:45295)
0809 02:13:54.444463 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:54.444465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:54.444516 (+ 51us) server_negotiation.cc:407] Connection header received
0809 02:13:54.444590 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:54.444591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:54.444599 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:54.444644 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:54.445175 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:54.445732 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:54.446433 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:54.446679 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:54.446780 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:54.448224 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:54.448225 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:54.448225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:54.448232 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:54.449859 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:54.450227 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:54.450229 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:54.450229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:54.450290 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:54.450489 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:54.450490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:54.450491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:54.450609 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:54.450708 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:54.450820 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 02:13:54.450899 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:13:54.464963 26848 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:13:54.465588 26848 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:13:54.470000 26848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7aae81c4bb854a9b999f49ff2ab52d5d: Generated new TSK 0
I20260809 02:13:54.470201 26848 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:13:54.474841 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 26762
2026-08-09T02:13:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:13:54 dist-test-slave-p3t5 krb5kdc[26866](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:13:54 dist-test-slave-p3t5 krb5kdc[26866](info): set up 2 sockets
Aug 09 02:13:54 dist-test-slave-p3t5 krb5kdc[26866](info): commencing operation
krb5kdc: starting...
W20260809 02:13:55.549991 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:13:55 dist-test-slave-p3t5 krb5kdc[26866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:13:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:13:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:13:55.686419 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:55.772715 26882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:55.772954 26882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:55.772979 26882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:55.774235 26882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:13:55.774284 26882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:55.774299 26882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:55.774312 26882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:13:55.774324 26882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:13:55.775878 26882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:55.776153 26882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:55.776414 26882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:13:55.781713 26882 server_base.cc:1034] running on GCE node
W20260809 02:13:55.781731 26887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:55.781761 26890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:55.781883 26888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:55.782234 26882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:55.782581 26882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:55.783761 26882 hybrid_clock.cc:648] HybridClock initialized: now 1786241635783735 us; error 49 us; skew 500 ppm
Aug 09 02:13:55 dist-test-slave-p3t5 krb5kdc[26866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241635, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:55.792430 26882 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:13:55.792899 26882 webserver.cc:460] Webserver started at http://127.0.0.1:38645/ using document root <none> and password file <none>
I20260809 02:13:55.793133 26882 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:55.793180 26882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:55.793293 26882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:55.794478 26882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "5a8f86330bba49baa9516211df1cbed8"
format_stamp: "Formatted at 2026-08-09 02:13:55 on dist-test-slave-p3t5"
server_key: "59d409e34670c852220dad978be809c7"
server_key_iv: "b6632bfba2afa2ee82e1983f1fac178a"
server_key_version: "encryptionkey@0"
I20260809 02:13:55.794847 26882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "5a8f86330bba49baa9516211df1cbed8"
format_stamp: "Formatted at 2026-08-09 02:13:55 on dist-test-slave-p3t5"
server_key: "59d409e34670c852220dad978be809c7"
server_key_iv: "b6632bfba2afa2ee82e1983f1fac178a"
server_key_version: "encryptionkey@0"
I20260809 02:13:55.796319 26882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:13:55.797132 26897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:55.797364 26882 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:13:55.797472 26882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "5a8f86330bba49baa9516211df1cbed8"
format_stamp: "Formatted at 2026-08-09 02:13:55 on dist-test-slave-p3t5"
server_key: "59d409e34670c852220dad978be809c7"
server_key_iv: "b6632bfba2afa2ee82e1983f1fac178a"
server_key_version: "encryptionkey@0"
I20260809 02:13:55.797554 26882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:55.809118 26882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:55.811816 26882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:55.811992 26882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:55.816525 26882 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39323
I20260809 02:13:55.816609 26950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39323 every 8 connection(s)
I20260809 02:13:55.816987 26882 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:13:55.818118 26951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:55.820452 26951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8: Bootstrap starting.
I20260809 02:13:55.821022 26951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:55.821283 26951 log.cc:824] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:55.822139 26951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8: No bootstrap required, opened a new log
I20260809 02:13:55.822690 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 26882
I20260809 02:13:55.822820 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:13:55.823024 5158 external_mini_cluster.cc:1428] Setting key 73fe23c96c5ae278082787bda1c223ed
I20260809 02:13:55.824846 26951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8f86330bba49baa9516211df1cbed8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } }
I20260809 02:13:55.825006 26951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:55.825030 26951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a8f86330bba49baa9516211df1cbed8, State: Initialized, Role: FOLLOWER
I20260809 02:13:55.825179 26951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8f86330bba49baa9516211df1cbed8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } }
I20260809 02:13:55.825255 26951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:13:55.825302 26951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:13:55.825361 26951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:55.826071 26951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8f86330bba49baa9516211df1cbed8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } }
I20260809 02:13:55.826198 26951 leader_election.cc:302] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a8f86330bba49baa9516211df1cbed8; no voters:
I20260809 02:13:55.826443 26951 leader_election.cc:288] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:13:55.826473 26958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:55.826598 26958 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [term 1 LEADER]: Becoming Leader. State: Replica: 5a8f86330bba49baa9516211df1cbed8, State: Running, Role: LEADER
I20260809 02:13:55.826694 26958 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8f86330bba49baa9516211df1cbed8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } }
I20260809 02:13:55.826763 26951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:13:55.827083 26958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a8f86330bba49baa9516211df1cbed8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8f86330bba49baa9516211df1cbed8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } } }
I20260809 02:13:55.827167 26958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:55.827304 26958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a8f86330bba49baa9516211df1cbed8. Latest consensus state: current_term: 1 leader_uuid: "5a8f86330bba49baa9516211df1cbed8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8f86330bba49baa9516211df1cbed8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } } }
I20260809 02:13:55.827404 26958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8 [sys.catalog]: This master's current role is: LEADER
W20260809 02:13:55.828953 26974 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:13:55.829030 26974 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:13:55.829116 26963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:13:55.829371 26963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:13:55.829576 26957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:55.823612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46391 (local address 127.0.0.1:39323)
0809 02:13:55.823830 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:55.823838 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:55.823869 (+ 31us) server_negotiation.cc:407] Connection header received
0809 02:13:55.824275 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:55.824283 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:55.824332 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:55.824469 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:55.824760 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:55.826196 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:55.827264 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:55.827750 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:55.827885 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:55.828640 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:55.828644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:55.828644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:55.828887 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:55.828932 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:55.828948 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:55.829012 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:55.829228 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 02:13:55.829293 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":116,"threads_started":1}
I20260809 02:13:55.830511 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:13:55.831146 26963 catalog_manager.cc:1347] Generated new cluster ID: 6282236d8b3941ea9029011e3443077e
I20260809 02:13:55.831220 26963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:13:55.835920 26957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:55.832088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57759 (local address 127.0.0.1:39323)
0809 02:13:55.832135 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:55.832137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:55.832231 (+ 94us) server_negotiation.cc:407] Connection header received
0809 02:13:55.832282 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:55.832283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:55.832295 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:55.832367 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:55.832519 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:55.833261 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:55.834432 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:55.834746 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:55.834829 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:55.835616 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:55.835618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:55.835619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:55.835624 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:55.835644 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:55.835649 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:55.835723 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:55.835776 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 02:13:55.835809 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 02:13:55.866142 26963 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:13:55.866945 26963 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:13:55.878646 26963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a8f86330bba49baa9516211df1cbed8: Generated new TSK 0
I20260809 02:13:55.878881 26963 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:13:55.890323 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 26882
2026-08-09T02:13:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1415 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:13:55 dist-test-slave-p3t5 krb5kdc[26987](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:13:55 dist-test-slave-p3t5 krb5kdc[26987](info): set up 2 sockets
Aug 09 02:13:55 dist-test-slave-p3t5 krb5kdc[26987](info): commencing operation
krb5kdc: starting...
W20260809 02:13:57.073975 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:13:57 dist-test-slave-p3t5 krb5kdc[26987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:13:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:13:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:13:57.206420 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40815
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:57.292549 27002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:57.292806 27002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:57.292850 27002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:57.294085 27002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:13:57.294159 27002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:57.294184 27002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:57.294215 27002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:13:57.294245 27002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:13:57.295746 27002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:57.296043 27002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:57.296280 27002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:57.301378 27008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:57.301410 27009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:57.301407 27011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:57.301438 27002 server_base.cc:1034] running on GCE node
I20260809 02:13:57.301939 27002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:57.302245 27002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:57.303404 27002 hybrid_clock.cc:648] HybridClock initialized: now 1786241637303375 us; error 41 us; skew 500 ppm
Aug 09 02:13:57 dist-test-slave-p3t5 krb5kdc[26987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241637, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:57.306115 27002 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:13:57.306602 27002 webserver.cc:460] Webserver started at http://127.0.0.1:42513/ using document root <none> and password file <none>
I20260809 02:13:57.306859 27002 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:57.306941 27002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:57.307101 27002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:57.308238 27002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "3085dc0fb4f348a7a73c8c69b23e0a7c"
format_stamp: "Formatted at 2026-08-09 02:13:57 on dist-test-slave-p3t5"
server_key: "cd662023f130e47b5ccf299e44e23376"
server_key_iv: "4034c4ac072dbff2865f74da0cd0721c"
server_key_version: "encryptionkey@0"
I20260809 02:13:57.308629 27002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "3085dc0fb4f348a7a73c8c69b23e0a7c"
format_stamp: "Formatted at 2026-08-09 02:13:57 on dist-test-slave-p3t5"
server_key: "cd662023f130e47b5ccf299e44e23376"
server_key_iv: "4034c4ac072dbff2865f74da0cd0721c"
server_key_version: "encryptionkey@0"
I20260809 02:13:57.310166 27002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:13:57.310983 27019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:57.311201 27002 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:13:57.311293 27002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "3085dc0fb4f348a7a73c8c69b23e0a7c"
format_stamp: "Formatted at 2026-08-09 02:13:57 on dist-test-slave-p3t5"
server_key: "cd662023f130e47b5ccf299e44e23376"
server_key_iv: "4034c4ac072dbff2865f74da0cd0721c"
server_key_version: "encryptionkey@0"
I20260809 02:13:57.311358 27002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:57.314903 27002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:57.316908 27002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:57.317023 27002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:57.321359 27002 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40815
I20260809 02:13:57.321396 27076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40815 every 8 connection(s)
I20260809 02:13:57.321847 27002 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:13:57.322006 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 27002
I20260809 02:13:57.322135 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:13:57.322361 5158 external_mini_cluster.cc:1428] Setting key e74c0a09db1ace5176e503b46ec8195c
I20260809 02:13:57.323235 27077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:57.325315 27080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:57.322992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37539 (local address 127.0.0.1:40815)
0809 02:13:57.323225 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:57.323236 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:57.323273 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:13:57.323907 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:57.323916 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:57.323952 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:57.324118 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:57.324861 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:57.324865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:57.324865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:57.324870 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:57.324920 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:57.324937 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:57.325011 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:57.325072 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 02:13:57.325116 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":113,"threads_started":1}
I20260809 02:13:57.326406 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:13:57.326692 27077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c: Bootstrap starting.
I20260809 02:13:57.327517 27077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:57.327925 27077 log.cc:824] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:57.328744 27077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c: No bootstrap required, opened a new log
I20260809 02:13:57.330642 27080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:57.328220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33261 (local address 127.0.0.1:40815)
0809 02:13:57.328264 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:57.328266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:57.328423 (+ 157us) server_negotiation.cc:407] Connection header received
0809 02:13:57.329793 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:57.329794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:57.329798 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:57.329841 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:57.330451 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:57.330452 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:57.330452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:13:57.330456 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:57.330469 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:13:57.330474 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:57.330519 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:57.330584 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 02:13:57.330596 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:13:57.331146 27077 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3085dc0fb4f348a7a73c8c69b23e0a7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40815 } }
I20260809 02:13:57.331321 27077 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:57.331347 27077 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3085dc0fb4f348a7a73c8c69b23e0a7c, State: Initialized, Role: FOLLOWER
I20260809 02:13:57.331491 27077 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3085dc0fb4f348a7a73c8c69b23e0a7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40815 } }
I20260809 02:13:57.331575 27077 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:13:57.331598 27077 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:13:57.331650 27077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:57.332230 27077 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3085dc0fb4f348a7a73c8c69b23e0a7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40815 } }
I20260809 02:13:57.332341 27077 leader_election.cc:302] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3085dc0fb4f348a7a73c8c69b23e0a7c; no voters:
I20260809 02:13:57.332521 27077 leader_election.cc:288] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:13:57.332576 27087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:57.332804 27077 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:13:57.332856 27087 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [term 1 LEADER]: Becoming Leader. State: Replica: 3085dc0fb4f348a7a73c8c69b23e0a7c, State: Running, Role: LEADER
I20260809 02:13:57.333019 27087 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3085dc0fb4f348a7a73c8c69b23e0a7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40815 } }
I20260809 02:13:57.333432 27089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3085dc0fb4f348a7a73c8c69b23e0a7c. Latest consensus state: current_term: 1 leader_uuid: "3085dc0fb4f348a7a73c8c69b23e0a7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3085dc0fb4f348a7a73c8c69b23e0a7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40815 } } }
I20260809 02:13:57.333527 27089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:57.333783 27088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3085dc0fb4f348a7a73c8c69b23e0a7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3085dc0fb4f348a7a73c8c69b23e0a7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40815 } } }
I20260809 02:13:57.333810 27099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:13:57.333932 27088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:57.334662 27099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:13:57.336383 27099 catalog_manager.cc:1347] Generated new cluster ID: f5577dffe0194701a2de7b8b5c58ac82
I20260809 02:13:57.336449 27099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:13:57.351991 27099 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:13:57.352214 27099 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:13:57.354370 27099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3085dc0fb4f348a7a73c8c69b23e0a7c: Generated new TSK 0
I20260809 02:13:57.354585 27099 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:13:57.356856 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 27002
2026-08-09T02:13:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1466 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:13:57 dist-test-slave-p3t5 krb5kdc[27111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:13:57 dist-test-slave-p3t5 krb5kdc[27111](info): set up 2 sockets
Aug 09 02:13:57 dist-test-slave-p3t5 krb5kdc[27111](info): commencing operation
krb5kdc: starting...
W20260809 02:13:58.438019 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:13:58 dist-test-slave-p3t5 krb5kdc[27111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:13:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:13:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:13:58.614807 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41555
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:13:58.754747 27125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:13:58.755242 27125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:13:58.755343 27125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:13:58.757328 27125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:13:58.757452 27125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:13:58.757520 27125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:13:58.757591 27125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:13:58.757683 27125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:13:58.760286 27125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:13:58.760910 27125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:13:58.761374 27125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:13:58.770216 27133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:58.770303 27131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:13:58.770246 27130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:13:58.771426 27125 server_base.cc:1034] running on GCE node
I20260809 02:13:58.771800 27125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:13:58.772225 27125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:13:58.773437 27125 hybrid_clock.cc:648] HybridClock initialized: now 1786241638773404 us; error 45 us; skew 500 ppm
Aug 09 02:13:58 dist-test-slave-p3t5 krb5kdc[27111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241638, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:13:58.777842 27125 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:13:58.778713 27125 webserver.cc:460] Webserver started at http://127.0.0.1:33257/ using document root <none> and password file <none>
I20260809 02:13:58.779119 27125 fs_manager.cc:359] Metadata directory not provided
I20260809 02:13:58.779263 27125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:13:58.779506 27125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:13:58.781580 27125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "2f4d07c70a7e4fb58ba09d6961aea927"
format_stamp: "Formatted at 2026-08-09 02:13:58 on dist-test-slave-p3t5"
server_key: "d24136124f1a6d8229f05f78d0d32d8b"
server_key_iv: "1e9841cc838f686bbb741dc58c3644d6"
server_key_version: "encryptionkey@0"
I20260809 02:13:58.782605 27125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "2f4d07c70a7e4fb58ba09d6961aea927"
format_stamp: "Formatted at 2026-08-09 02:13:58 on dist-test-slave-p3t5"
server_key: "d24136124f1a6d8229f05f78d0d32d8b"
server_key_iv: "1e9841cc838f686bbb741dc58c3644d6"
server_key_version: "encryptionkey@0"
I20260809 02:13:58.785581 27125 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:13:58.787390 27140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:13:58.787710 27125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:13:58.787943 27125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "2f4d07c70a7e4fb58ba09d6961aea927"
format_stamp: "Formatted at 2026-08-09 02:13:58 on dist-test-slave-p3t5"
server_key: "d24136124f1a6d8229f05f78d0d32d8b"
server_key_iv: "1e9841cc838f686bbb741dc58c3644d6"
server_key_version: "encryptionkey@0"
I20260809 02:13:58.788121 27125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:13:58.803382 27125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:13:58.807195 27125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:13:58.807483 27125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:13:58.815083 27125 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41555
I20260809 02:13:58.815174 27196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41555 every 8 connection(s)
I20260809 02:13:58.816169 27125 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:13:58.817900 27197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:13:58.822526 27197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927: Bootstrap starting.
I20260809 02:13:58.823571 27197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927: Neither blocks nor log segments found. Creating new log.
I20260809 02:13:58.824093 27197 log.cc:824] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927: Log is configured to *not* fsync() on all Append() calls
I20260809 02:13:58.824700 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 27125
I20260809 02:13:58.824942 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:13:58.825284 5158 external_mini_cluster.cc:1428] Setting key f86b1c38653047a803da7552faf907a1
I20260809 02:13:58.825667 27197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927: No bootstrap required, opened a new log
I20260809 02:13:58.829779 27197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4d07c70a7e4fb58ba09d6961aea927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } }
I20260809 02:13:58.830183 27197 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:13:58.830281 27197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f4d07c70a7e4fb58ba09d6961aea927, State: Initialized, Role: FOLLOWER
I20260809 02:13:58.830487 27197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4d07c70a7e4fb58ba09d6961aea927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } }
I20260809 02:13:58.830636 27197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:13:58.830703 27197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:13:58.830961 27197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:13:58.832365 27197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4d07c70a7e4fb58ba09d6961aea927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } }
I20260809 02:13:58.832559 27197 leader_election.cc:302] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f4d07c70a7e4fb58ba09d6961aea927; no voters:
I20260809 02:13:58.833024 27197 leader_election.cc:288] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:13:58.833048 27202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:13:58.833256 27202 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [term 1 LEADER]: Becoming Leader. State: Replica: 2f4d07c70a7e4fb58ba09d6961aea927, State: Running, Role: LEADER
I20260809 02:13:58.833549 27197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:13:58.833531 27202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4d07c70a7e4fb58ba09d6961aea927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } }
I20260809 02:13:58.837347 27203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f4d07c70a7e4fb58ba09d6961aea927" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4d07c70a7e4fb58ba09d6961aea927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } } }
I20260809 02:13:58.837507 27203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:58.838083 27217 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:13:58.838121 27204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f4d07c70a7e4fb58ba09d6961aea927. Latest consensus state: current_term: 1 leader_uuid: "2f4d07c70a7e4fb58ba09d6961aea927" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4d07c70a7e4fb58ba09d6961aea927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } } }
I20260809 02:13:58.838274 27204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927 [sys.catalog]: This master's current role is: LEADER
I20260809 02:13:58.839083 27217 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 02:13:58 dist-test-slave-p3t5 krb5kdc[27111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 02:13:58.842952 27217 catalog_manager.cc:1347] Generated new cluster ID: 90b7bed9f6dc4b00aae723dc9ff81b18
I20260809 02:13:58.843067 27217 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:13:58.848297 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:58.826323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:54189 (local address 127.0.0.1:41555)
0809 02:13:58.826628 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:58.826639 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:58.826712 (+ 73us) server_negotiation.cc:407] Connection header received
0809 02:13:58.828126 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:58.828136 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:58.828260 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:58.828548 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:58.829488 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:58.832359 (+ 2871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:58.834066 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:58.835373 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:58.835580 (+ 207us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:58.840815 (+ 5235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:58.840820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:58.840820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:58.840848 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:58.844748 (+ 3900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:58.845803 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:58.845806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:58.845807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:58.846006 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:58.846452 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:58.846453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:58.846454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:58.847066 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:58.847477 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:58.847693 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 02:13:58.847874 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":148,"threads_started":1}
I20260809 02:13:58.850332 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:13:58.865732 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:13:58.852828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:42549 (local address 127.0.0.1:41555)
0809 02:13:58.852892 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 02:13:58.852894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:13:58.853100 (+ 206us) server_negotiation.cc:407] Connection header received
0809 02:13:58.853218 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:13:58.853219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:13:58.853231 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:13:58.853343 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:13:58.854608 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:58.855678 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:58.857282 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:13:58.857810 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:13:58.857904 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:13:58.860664 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:13:58.860666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:13:58.860666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:13:58.860683 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:13:58.863545 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:58.864297 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:58.864298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:58.864299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:58.864389 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:13:58.864761 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:13:58.864762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:13:58.864762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:13:58.864970 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:13:58.865121 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 02:13:58.865421 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 02:13:58.865535 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 02:13:58.885150 27217 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:13:58.886428 27217 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:13:58.896464 27217 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f4d07c70a7e4fb58ba09d6961aea927: Generated new TSK 0
I20260809 02:13:58.896888 27217 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:13:58.903367 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 27125
W20260809 02:13:58.913829 27109 connection.cc:535] client connection to 127.0.0.1:41555 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41555): Connection reset by peer (error 104)
2026-08-09T02:13:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1551 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:13:58 dist-test-slave-p3t5 krb5kdc[27228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:13:58 dist-test-slave-p3t5 krb5kdc[27228](info): set up 2 sockets
Aug 09 02:13:58 dist-test-slave-p3t5 krb5kdc[27228](info): commencing operation
krb5kdc: starting...
W20260809 02:14:00.319087 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.388s user 0.008s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:14:00 dist-test-slave-p3t5 krb5kdc[27228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:14:00.456897 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40005
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:00.541790 27242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:00.542073 27242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:00.542114 27242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:00.543316 27242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:00.543407 27242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:00.543434 27242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:00.543465 27242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:00.543484 27242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:00.545048 27242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:00.545356 27242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:00.545657 27242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:00.550937 27248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:00.550895 27247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:00.550925 27242 server_base.cc:1034] running on GCE node
W20260809 02:14:00.550880 27250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:00.551462 27242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:00.551788 27242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:00.552977 27242 hybrid_clock.cc:648] HybridClock initialized: now 1786241640552961 us; error 30 us; skew 500 ppm
Aug 09 02:14:00 dist-test-slave-p3t5 krb5kdc[27228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241640, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:00.555841 27242 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:14:00.556361 27242 webserver.cc:460] Webserver started at http://127.0.0.1:38821/ using document root <none> and password file <none>
I20260809 02:14:00.556643 27242 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:00.556722 27242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:00.556885 27242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:00.558209 27242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "044120aaf1b24482bd5a431e4a452524"
format_stamp: "Formatted at 2026-08-09 02:14:00 on dist-test-slave-p3t5"
server_key: "844149bc49a3c358fa0efc10ed9e13f0"
server_key_iv: "bd01572a8de427203e2585e4a09655ac"
server_key_version: "encryptionkey@0"
I20260809 02:14:00.558615 27242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "044120aaf1b24482bd5a431e4a452524"
format_stamp: "Formatted at 2026-08-09 02:14:00 on dist-test-slave-p3t5"
server_key: "844149bc49a3c358fa0efc10ed9e13f0"
server_key_iv: "bd01572a8de427203e2585e4a09655ac"
server_key_version: "encryptionkey@0"
I20260809 02:14:00.560039 27242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:14:00.561005 27257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:00.561241 27242 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:00.561345 27242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "044120aaf1b24482bd5a431e4a452524"
format_stamp: "Formatted at 2026-08-09 02:14:00 on dist-test-slave-p3t5"
server_key: "844149bc49a3c358fa0efc10ed9e13f0"
server_key_iv: "bd01572a8de427203e2585e4a09655ac"
server_key_version: "encryptionkey@0"
I20260809 02:14:00.561429 27242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:00.586114 27242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:00.588423 27242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:00.588614 27242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:00.592998 27242 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40005
I20260809 02:14:00.593034 27311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40005 every 8 connection(s)
I20260809 02:14:00.593477 27242 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:00.594622 27312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:00.597180 27312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524: Bootstrap starting.
I20260809 02:14:00.597877 27312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:00.598201 27312 log.cc:824] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:00.599030 27312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524: No bootstrap required, opened a new log
I20260809 02:14:00.600719 27312 raft_consensus.cc:348] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "044120aaf1b24482bd5a431e4a452524" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40005 } }
I20260809 02:14:00.600875 27312 raft_consensus.cc:374] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:00.600899 27312 raft_consensus.cc:729] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 044120aaf1b24482bd5a431e4a452524, State: Initialized, Role: FOLLOWER
I20260809 02:14:00.601086 27312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "044120aaf1b24482bd5a431e4a452524" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40005 } }
I20260809 02:14:00.601152 27312 raft_consensus.cc:388] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:00.601174 27312 raft_consensus.cc:482] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:00.601245 27312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:00.601922 27312 raft_consensus.cc:504] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "044120aaf1b24482bd5a431e4a452524" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40005 } }
I20260809 02:14:00.602044 27312 leader_election.cc:302] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 044120aaf1b24482bd5a431e4a452524; no voters:
I20260809 02:14:00.602239 27312 leader_election.cc:288] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:00.602290 27315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:00.602555 27315 raft_consensus.cc:686] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [term 1 LEADER]: Becoming Leader. State: Replica: 044120aaf1b24482bd5a431e4a452524, State: Running, Role: LEADER
I20260809 02:14:00.602558 27312 sys_catalog.cc:564] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:00.602669 27315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "044120aaf1b24482bd5a431e4a452524" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40005 } }
I20260809 02:14:00.603089 27317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 044120aaf1b24482bd5a431e4a452524. Latest consensus state: current_term: 1 leader_uuid: "044120aaf1b24482bd5a431e4a452524" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "044120aaf1b24482bd5a431e4a452524" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40005 } } }
I20260809 02:14:00.603201 27317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:00.603091 27316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "044120aaf1b24482bd5a431e4a452524" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "044120aaf1b24482bd5a431e4a452524" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40005 } } }
I20260809 02:14:00.603536 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 27242
I20260809 02:14:00.603689 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:14:00.603914 27316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:00.603940 5158 external_mini_cluster.cc:1428] Setting key ae6b63966389e972d024d63ac7b439da
I20260809 02:14:00.603444 27319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:00.604521 27319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:00.606163 27319 catalog_manager.cc:1347] Generated new cluster ID: b0a7607ca3b44aa2923ec788ff0ebe11
I20260809 02:14:00.606221 27319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:00.610064 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:00.604643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:57393 (local address 127.0.0.1:40005)
0809 02:14:00.604825 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:00.604835 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:00.605235 (+ 400us) server_negotiation.cc:407] Connection header received
0809 02:14:00.605369 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:00.605375 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:00.605424 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:00.605591 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:00.606050 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:00.607090 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:00.607949 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:00.608399 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:00.608535 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:00.609340 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:00.609344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:00.609344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:00.609541 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:00.609575 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:00.609587 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:00.609719 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:00.609801 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 02:14:00.609834 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":102,"threads_started":1}
I20260809 02:14:00.611115 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:00.616894 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:00.612607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:40861 (local address 127.0.0.1:40005)
0809 02:14:00.612647 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:00.612649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:00.613884 (+ 1235us) server_negotiation.cc:407] Connection header received
0809 02:14:00.613965 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:00.613967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:00.613978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:00.614030 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:00.614216 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:00.614719 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:00.615508 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:00.615728 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:00.615823 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:00.616598 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:00.616600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:00.616600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:00.616605 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:00.616622 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:00.616627 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:00.616703 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:00.616783 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 02:14:00.616798 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 02:14:00.621376 27319 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:00.622002 27319 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:00.627183 27319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 044120aaf1b24482bd5a431e4a452524: Generated new TSK 0
I20260809 02:14:00.627421 27319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:14:00.630088 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 27242
W20260809 02:14:00.638265 27226 connection.cc:535] client connection to 127.0.0.1:40005 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40005): Connection reset by peer (error 104)
2026-08-09T02:14:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1722 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:00 dist-test-slave-p3t5 krb5kdc[27343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:00 dist-test-slave-p3t5 krb5kdc[27343](info): set up 2 sockets
Aug 09 02:14:00 dist-test-slave-p3t5 krb5kdc[27343](info): commencing operation
krb5kdc: starting...
W20260809 02:14:01.710624 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:14:01 dist-test-slave-p3t5 krb5kdc[27343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:14:01.854312 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:01.939606 27357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:01.939929 27357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:01.939978 27357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:01.941220 27357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:01.941318 27357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:01.941344 27357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:01.941366 27357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:01.941397 27357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:01.943174 27357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:01.943504 27357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:01.943759 27357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:01.949172 27364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:01.949172 27366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:01.949246 27363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:01.949609 27357 server_base.cc:1034] running on GCE node
I20260809 02:14:01.950002 27357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:01.950333 27357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:01.951517 27357 hybrid_clock.cc:648] HybridClock initialized: now 1786241641951490 us; error 65 us; skew 500 ppm
Aug 09 02:14:01 dist-test-slave-p3t5 krb5kdc[27343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241641, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:01.959450 27357 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:14:01.960037 27357 webserver.cc:460] Webserver started at http://127.0.0.1:43333/ using document root <none> and password file <none>
I20260809 02:14:01.960322 27357 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:01.960404 27357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:01.960566 27357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:01.961838 27357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "96c1ee9f4b3d4605933b20fe9214c891"
format_stamp: "Formatted at 2026-08-09 02:14:01 on dist-test-slave-p3t5"
server_key: "a442f2c910a1e6533a36b8af939c38c5"
server_key_iv: "6d3861b55037b89f8e5b3c152d66ec89"
server_key_version: "encryptionkey@0"
I20260809 02:14:01.962235 27357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "96c1ee9f4b3d4605933b20fe9214c891"
format_stamp: "Formatted at 2026-08-09 02:14:01 on dist-test-slave-p3t5"
server_key: "a442f2c910a1e6533a36b8af939c38c5"
server_key_iv: "6d3861b55037b89f8e5b3c152d66ec89"
server_key_version: "encryptionkey@0"
I20260809 02:14:01.963878 27357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:01.964807 27374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:01.965078 27357 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:01.965205 27357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "96c1ee9f4b3d4605933b20fe9214c891"
format_stamp: "Formatted at 2026-08-09 02:14:01 on dist-test-slave-p3t5"
server_key: "a442f2c910a1e6533a36b8af939c38c5"
server_key_iv: "6d3861b55037b89f8e5b3c152d66ec89"
server_key_version: "encryptionkey@0"
I20260809 02:14:01.965297 27357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:01.969257 27357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:01.972142 27357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:01.972343 27357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:01.977511 27357 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43695
I20260809 02:14:01.977551 27433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43695 every 8 connection(s)
I20260809 02:14:01.978004 27357 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:01.979112 27434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:01.980834 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 27357
I20260809 02:14:01.980960 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:14:01.981179 5158 external_mini_cluster.cc:1428] Setting key 8e68d8e33a8bcc79101c9285b9b612ef
I20260809 02:14:01.982071 27434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891: Bootstrap starting.
I20260809 02:14:01.982710 27434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:01.983096 27434 log.cc:824] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:01.984014 27434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891: No bootstrap required, opened a new log
I20260809 02:14:01.984193 27437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:01.981868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:46377 (local address 127.0.0.1:43695)
0809 02:14:01.982089 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:01.982097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:01.982154 (+ 57us) server_negotiation.cc:407] Connection header received
0809 02:14:01.982642 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:01.982650 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:01.982676 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:01.982876 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:01.983662 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:01.983667 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:01.983667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:01.983673 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:01.983722 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:01.983737 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:01.983841 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:01.983882 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 02:14:01.983927 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":90,"threads_started":1}
I20260809 02:14:01.985347 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:01.986286 27434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c1ee9f4b3d4605933b20fe9214c891" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43695 } }
I20260809 02:14:01.986492 27434 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:01.986539 27434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96c1ee9f4b3d4605933b20fe9214c891, State: Initialized, Role: FOLLOWER
I20260809 02:14:01.986668 27434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c1ee9f4b3d4605933b20fe9214c891" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43695 } }
I20260809 02:14:01.986809 27434 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:01.986855 27434 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:01.986912 27434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:01.987528 27434 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c1ee9f4b3d4605933b20fe9214c891" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43695 } }
I20260809 02:14:01.987664 27434 leader_election.cc:302] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96c1ee9f4b3d4605933b20fe9214c891; no voters:
I20260809 02:14:01.987895 27434 leader_election.cc:288] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:01.987924 27445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:01.988106 27445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [term 1 LEADER]: Becoming Leader. State: Replica: 96c1ee9f4b3d4605933b20fe9214c891, State: Running, Role: LEADER
I20260809 02:14:01.988299 27437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:01.986892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:53509 (local address 127.0.0.1:43695)
0809 02:14:01.986930 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:01.986932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:01.987117 (+ 185us) server_negotiation.cc:407] Connection header received
0809 02:14:01.987194 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:01.987195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:01.987201 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:01.987252 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:01.987997 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:01.987998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:01.987998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:01.988004 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:01.988019 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:01.988023 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:01.988070 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:01.988161 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 02:14:01.988176 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 02:14:01.988199 27445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c1ee9f4b3d4605933b20fe9214c891" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43695 } }
I20260809 02:14:01.988245 27434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:01.988914 27446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96c1ee9f4b3d4605933b20fe9214c891. Latest consensus state: current_term: 1 leader_uuid: "96c1ee9f4b3d4605933b20fe9214c891" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c1ee9f4b3d4605933b20fe9214c891" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43695 } } }
I20260809 02:14:01.989023 27446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:01.988955 27447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96c1ee9f4b3d4605933b20fe9214c891" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c1ee9f4b3d4605933b20fe9214c891" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43695 } } }
I20260809 02:14:01.989166 27447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:01.989409 27449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:01.989924 27449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:01.991619 27449 catalog_manager.cc:1347] Generated new cluster ID: 60c0bde7fd024ea88ac7314c4854bd69
I20260809 02:14:01.991667 27449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:02.003253 27449 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:02.003458 27449 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:02.008523 27449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96c1ee9f4b3d4605933b20fe9214c891: Generated new TSK 0
I20260809 02:14:02.008749 27449 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:14:02.017560 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 27357
2026-08-09T02:14:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:02 dist-test-slave-p3t5 krb5kdc[27468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:02 dist-test-slave-p3t5 krb5kdc[27468](info): set up 2 sockets
Aug 09 02:14:02 dist-test-slave-p3t5 krb5kdc[27468](info): commencing operation
krb5kdc: starting...
W20260809 02:14:03.101941 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:14:03 dist-test-slave-p3t5 krb5kdc[27468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:14:03.241122 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36741
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:03.327519 27481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:03.327759 27481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:03.327802 27481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:03.329030 27481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:03.329077 27481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:03.329090 27481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:03.329103 27481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:03.329159 27481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:03.330806 27481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36741
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:03.331097 27481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:03.331351 27481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:14:03.337214 27481 server_base.cc:1034] running on GCE node
W20260809 02:14:03.337167 27487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:03.337199 27486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:03.337191 27489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:03.337857 27481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:03.338195 27481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:03.339381 27481 hybrid_clock.cc:648] HybridClock initialized: now 1786241643339378 us; error 40 us; skew 500 ppm
Aug 09 02:14:03 dist-test-slave-p3t5 krb5kdc[27468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241643, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:03.342381 27481 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:14:03.342948 27481 webserver.cc:460] Webserver started at http://127.0.0.1:40685/ using document root <none> and password file <none>
I20260809 02:14:03.343211 27481 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:03.343289 27481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:03.343456 27481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:03.344573 27481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "4136d336f75144298db5712c7354ad8a"
format_stamp: "Formatted at 2026-08-09 02:14:03 on dist-test-slave-p3t5"
server_key: "41e288bf1cd578d819bbbab96b4adf6c"
server_key_iv: "cbf5e2b82a1eee6caa551c6a071dcd77"
server_key_version: "encryptionkey@0"
I20260809 02:14:03.344970 27481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "4136d336f75144298db5712c7354ad8a"
format_stamp: "Formatted at 2026-08-09 02:14:03 on dist-test-slave-p3t5"
server_key: "41e288bf1cd578d819bbbab96b4adf6c"
server_key_iv: "cbf5e2b82a1eee6caa551c6a071dcd77"
server_key_version: "encryptionkey@0"
I20260809 02:14:03.346627 27481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:14:03.347488 27496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:03.347805 27481 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:03.347954 27481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "4136d336f75144298db5712c7354ad8a"
format_stamp: "Formatted at 2026-08-09 02:14:03 on dist-test-slave-p3t5"
server_key: "41e288bf1cd578d819bbbab96b4adf6c"
server_key_iv: "cbf5e2b82a1eee6caa551c6a071dcd77"
server_key_version: "encryptionkey@0"
I20260809 02:14:03.348052 27481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:03.367143 27481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:03.369256 27481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:03.369457 27481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:03.374002 27481 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36741
I20260809 02:14:03.374045 27553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36741 every 8 connection(s)
I20260809 02:14:03.374459 27481 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:03.375515 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:03.377555 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 27481
I20260809 02:14:03.377702 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:14:03.377919 5158 external_mini_cluster.cc:1428] Setting key 6bc8a29536ff52f2339190934160f546
I20260809 02:14:03.378322 27554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a: Bootstrap starting.
I20260809 02:14:03.379182 27554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:03.379539 27554 log.cc:824] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:03.380425 27554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a: No bootstrap required, opened a new log
I20260809 02:14:03.382858 27554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4136d336f75144298db5712c7354ad8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36741 } }
I20260809 02:14:03.383041 27554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:03.383076 27554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4136d336f75144298db5712c7354ad8a, State: Initialized, Role: FOLLOWER
I20260809 02:14:03.383200 27554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4136d336f75144298db5712c7354ad8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36741 } }
I20260809 02:14:03.383275 27554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:03.383306 27554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:03.383344 27554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:03.384173 27554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4136d336f75144298db5712c7354ad8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36741 } }
I20260809 02:14:03.384306 27554 leader_election.cc:302] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4136d336f75144298db5712c7354ad8a; no voters:
I20260809 02:14:03.384510 27554 leader_election.cc:288] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:03.384596 27559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:03.384799 27559 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [term 1 LEADER]: Becoming Leader. State: Replica: 4136d336f75144298db5712c7354ad8a, State: Running, Role: LEADER
I20260809 02:14:03.384821 27554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:03.384896 27559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4136d336f75144298db5712c7354ad8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36741 } }
I20260809 02:14:03.385264 27560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4136d336f75144298db5712c7354ad8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4136d336f75144298db5712c7354ad8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36741 } } }
I20260809 02:14:03.385253 27559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4136d336f75144298db5712c7354ad8a. Latest consensus state: current_term: 1 leader_uuid: "4136d336f75144298db5712c7354ad8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4136d336f75144298db5712c7354ad8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36741 } } }
I20260809 02:14:03.385421 27560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:03.385429 27559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:03.385747 27568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 02:14:03 dist-test-slave-p3t5 krb5kdc[27468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 02:14:03.386297 27568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:03.388230 27568 catalog_manager.cc:1347] Generated new cluster ID: c0e4ea84f12b4ed6b1b5fe9c779b0be4
I20260809 02:14:03.388288 27568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:03.392833 27557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:03.378497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:52185 (local address 127.0.0.1:36741)
0809 02:14:03.378717 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:03.378728 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:03.378767 (+ 39us) server_negotiation.cc:407] Connection header received
0809 02:14:03.379427 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:03.379436 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:03.379514 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:03.379677 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:03.380519 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:03.381960 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:03.382737 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:03.383379 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:03.383498 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:03.387123 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:03.387128 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:03.387128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:03.387143 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:03.390638 (+ 3495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:03.391186 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:03.391189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:03.391190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:03.391300 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:03.391575 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:03.391576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:03.391577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:03.391890 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:03.392430 (+ 540us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:03.392511 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 02:14:03.392612 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":102,"threads_started":1}
I20260809 02:14:03.393988 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:03.403595 27557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:03.395714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:39353 (local address 127.0.0.1:36741)
0809 02:14:03.395753 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:03.395755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:03.395918 (+ 163us) server_negotiation.cc:407] Connection header received
0809 02:14:03.395965 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:03.395965 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:03.395975 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:03.396027 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:03.396719 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:03.397477 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:03.398567 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:03.398912 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:03.398977 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:03.400557 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:03.400559 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:03.400559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:03.400566 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:03.402222 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:03.402685 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:03.402686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:03.402687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:03.402750 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:03.403003 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:03.403003 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:03.403004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:03.403138 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:03.403240 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:03.403432 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 02:14:03.403504 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 02:14:03.405000 27568 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:03.405781 27568 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:03.412652 27568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4136d336f75144298db5712c7354ad8a: Generated new TSK 0
I20260809 02:14:03.412892 27568 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:14:03.419407 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 27481
W20260809 02:14:03.426654 27466 connection.cc:535] client connection to 127.0.0.1:36741 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36741): Connection reset by peer (error 104)
2026-08-09T02:14:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1402 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:03 dist-test-slave-p3t5 krb5kdc[27587](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:03 dist-test-slave-p3t5 krb5kdc[27587](info): set up 2 sockets
Aug 09 02:14:03 dist-test-slave-p3t5 krb5kdc[27587](info): commencing operation
krb5kdc: starting...
W20260809 02:14:04.538127 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:14:04 dist-test-slave-p3t5 krb5kdc[27587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:14:04.677904 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41199
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:04.763170 27600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:04.763476 27600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:04.763521 27600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:04.764761 27600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:04.764860 27600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:04.764886 27600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:04.764907 27600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:04.764935 27600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:04.766585 27600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41199
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:04.766932 27600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:04.767198 27600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:04.772799 27605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:04.772891 27606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:04.772893 27608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:04.773036 27600 server_base.cc:1034] running on GCE node
I20260809 02:14:04.773290 27600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:04.773648 27600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:04.774827 27600 hybrid_clock.cc:648] HybridClock initialized: now 1786241644774799 us; error 47 us; skew 500 ppm
Aug 09 02:14:04 dist-test-slave-p3t5 krb5kdc[27587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241644, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:04.777541 27600 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:14:04.778106 27600 webserver.cc:460] Webserver started at http://127.0.0.1:38771/ using document root <none> and password file <none>
I20260809 02:14:04.778378 27600 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:04.778462 27600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:04.778627 27600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:04.779757 27600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "f1fe40d595504ab5aeecae4a5957d905"
format_stamp: "Formatted at 2026-08-09 02:14:04 on dist-test-slave-p3t5"
server_key: "92d5deaaf401ec3954638dfa9c3f11f0"
server_key_iv: "a5354f6f6c8f5e8f42140604c7d51a7b"
server_key_version: "encryptionkey@0"
I20260809 02:14:04.780145 27600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "f1fe40d595504ab5aeecae4a5957d905"
format_stamp: "Formatted at 2026-08-09 02:14:04 on dist-test-slave-p3t5"
server_key: "92d5deaaf401ec3954638dfa9c3f11f0"
server_key_iv: "a5354f6f6c8f5e8f42140604c7d51a7b"
server_key_version: "encryptionkey@0"
I20260809 02:14:04.781891 27600 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:14:04.782889 27616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:04.783253 27600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:04.783377 27600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "f1fe40d595504ab5aeecae4a5957d905"
format_stamp: "Formatted at 2026-08-09 02:14:04 on dist-test-slave-p3t5"
server_key: "92d5deaaf401ec3954638dfa9c3f11f0"
server_key_iv: "a5354f6f6c8f5e8f42140604c7d51a7b"
server_key_version: "encryptionkey@0"
I20260809 02:14:04.783476 27600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:04.793576 27600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:04.795823 27600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:04.795940 27600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:04.800798 27600 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41199
I20260809 02:14:04.800833 27670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41199 every 8 connection(s)
I20260809 02:14:04.801239 27600 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:04.802314 27671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:04.803992 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 27600
I20260809 02:14:04.804107 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:14:04.804334 5158 external_mini_cluster.cc:1428] Setting key b8fff480de2bc6137e49a7d0b6153bda
I20260809 02:14:04.804994 27671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905: Bootstrap starting.
I20260809 02:14:04.805778 27671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:04.806151 27671 log.cc:824] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:04.807004 27671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905: No bootstrap required, opened a new log
I20260809 02:14:04.808933 27671 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1fe40d595504ab5aeecae4a5957d905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41199 } }
I20260809 02:14:04.809110 27671 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:04.809162 27671 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1fe40d595504ab5aeecae4a5957d905, State: Initialized, Role: FOLLOWER
I20260809 02:14:04.809671 27671 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1fe40d595504ab5aeecae4a5957d905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41199 } }
I20260809 02:14:04.809798 27671 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:04.809856 27671 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:04.809926 27671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:04.810454 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:04.804939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:32787 (local address 127.0.0.1:41199)
0809 02:14:04.805194 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:04.805204 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:04.805236 (+ 32us) server_negotiation.cc:407] Connection header received
0809 02:14:04.805933 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:04.805946 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:04.806027 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:04.806193 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:04.806482 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:04.807565 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:04.808369 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:04.808797 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:04.808942 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:04.809713 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:04.809716 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:04.809717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:04.809981 (+ 264us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:04.810025 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:04.810042 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:04.810151 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:04.810235 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 02:14:04.810264 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":117,"threads_started":1}
I20260809 02:14:04.810685 27671 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1fe40d595504ab5aeecae4a5957d905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41199 } }
I20260809 02:14:04.810817 27671 leader_election.cc:302] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1fe40d595504ab5aeecae4a5957d905; no voters:
I20260809 02:14:04.811050 27671 leader_election.cc:288] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:04.811097 27676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:04.811262 27676 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [term 1 LEADER]: Becoming Leader. State: Replica: f1fe40d595504ab5aeecae4a5957d905, State: Running, Role: LEADER
I20260809 02:14:04.811317 27671 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:04.811398 27676 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1fe40d595504ab5aeecae4a5957d905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41199 } }
I20260809 02:14:04.811478 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:04.812744 27677 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1fe40d595504ab5aeecae4a5957d905. Latest consensus state: current_term: 1 leader_uuid: "f1fe40d595504ab5aeecae4a5957d905" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1fe40d595504ab5aeecae4a5957d905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41199 } } }
I20260809 02:14:04.812839 27677 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:04.813030 27678 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1fe40d595504ab5aeecae4a5957d905" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1fe40d595504ab5aeecae4a5957d905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41199 } } }
I20260809 02:14:04.813040 27698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:04.813170 27678 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:04.813807 27698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:04.817274 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:04.813247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:49677 (local address 127.0.0.1:41199)
0809 02:14:04.813283 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:04.813285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:04.813341 (+ 56us) server_negotiation.cc:407] Connection header received
0809 02:14:04.813445 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:04.813446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:04.813457 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:04.813510 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:04.813738 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:04.814497 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:04.815579 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:04.815998 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:04.816081 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:04.816993 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:04.816994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:04.816995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:04.817000 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:04.817015 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:04.817019 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:04.817079 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:04.817170 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 02:14:04.817180 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 02:14:04.821812 27698 catalog_manager.cc:1347] Generated new cluster ID: 79a08771d94246a09447ac9bf80c3f3e
I20260809 02:14:04.821913 27698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:04.826290 27698 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:04.826849 27698 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:04.829977 27698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1fe40d595504ab5aeecae4a5957d905: Generated new TSK 0
I20260809 02:14:04.830173 27698 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:14:04.839744 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 27600
W20260809 02:14:04.846081 27584 connection.cc:535] client connection to 127.0.0.1:41199 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41199): Connection reset by peer (error 104)
2026-08-09T02:14:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1419 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:04 dist-test-slave-p3t5 krb5kdc[27704](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:04 dist-test-slave-p3t5 krb5kdc[27704](info): set up 2 sockets
Aug 09 02:14:04 dist-test-slave-p3t5 krb5kdc[27704](info): commencing operation
krb5kdc: starting...
W20260809 02:14:05.918298 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:14:06 dist-test-slave-p3t5 krb5kdc[27704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:14:06.053844 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44777
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:06.139242 27717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:06.139539 27717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:06.139585 27717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:06.140810 27717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:06.140898 27717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:06.140923 27717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:06.140945 27717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:06.140980 27717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:06.142568 27717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44777
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:06.142879 27717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:06.143141 27717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:06.148454 27724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:06.148455 27725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:06.148535 27717 server_base.cc:1034] running on GCE node
W20260809 02:14:06.148496 27727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:06.149102 27717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:06.149443 27717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:06.150627 27717 hybrid_clock.cc:648] HybridClock initialized: now 1786241646150602 us; error 49 us; skew 500 ppm
Aug 09 02:14:06 dist-test-slave-p3t5 krb5kdc[27704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241646, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:06.153555 27717 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:14:06.154188 27717 webserver.cc:460] Webserver started at http://127.0.0.1:43517/ using document root <none> and password file <none>
I20260809 02:14:06.154500 27717 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:06.154632 27717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:06.154803 27717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:06.156349 27717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "89f7393d874645bcadd566c3732b8622"
format_stamp: "Formatted at 2026-08-09 02:14:06 on dist-test-slave-p3t5"
server_key: "4b884be7b6fcb26c7fe52829522579bc"
server_key_iv: "05696f168cf6db958aac704915020c54"
server_key_version: "encryptionkey@0"
I20260809 02:14:06.156802 27717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "89f7393d874645bcadd566c3732b8622"
format_stamp: "Formatted at 2026-08-09 02:14:06 on dist-test-slave-p3t5"
server_key: "4b884be7b6fcb26c7fe52829522579bc"
server_key_iv: "05696f168cf6db958aac704915020c54"
server_key_version: "encryptionkey@0"
I20260809 02:14:06.158483 27717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:06.159281 27734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:06.159552 27717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:06.159667 27717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "89f7393d874645bcadd566c3732b8622"
format_stamp: "Formatted at 2026-08-09 02:14:06 on dist-test-slave-p3t5"
server_key: "4b884be7b6fcb26c7fe52829522579bc"
server_key_iv: "05696f168cf6db958aac704915020c54"
server_key_version: "encryptionkey@0"
I20260809 02:14:06.159766 27717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:06.163393 27717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:06.165596 27717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:06.165808 27717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:06.170390 27717 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44777
I20260809 02:14:06.170426 27790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44777 every 8 connection(s)
I20260809 02:14:06.170794 27717 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:06.171844 27791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:06.174327 27791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622: Bootstrap starting.
I20260809 02:14:06.174880 27791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:06.175146 27791 log.cc:824] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:06.175860 27791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622: No bootstrap required, opened a new log
I20260809 02:14:06.177453 27791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f7393d874645bcadd566c3732b8622" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44777 } }
I20260809 02:14:06.177594 27791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:06.177687 27791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f7393d874645bcadd566c3732b8622, State: Initialized, Role: FOLLOWER
I20260809 02:14:06.177817 27791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f7393d874645bcadd566c3732b8622" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44777 } }
I20260809 02:14:06.177873 27791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:06.177894 27791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:06.177922 27791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:06.178429 27791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f7393d874645bcadd566c3732b8622" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44777 } }
I20260809 02:14:06.178516 27791 leader_election.cc:302] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89f7393d874645bcadd566c3732b8622; no voters:
I20260809 02:14:06.178642 27791 leader_election.cc:288] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:06.178718 27796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:06.178913 27796 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [term 1 LEADER]: Becoming Leader. State: Replica: 89f7393d874645bcadd566c3732b8622, State: Running, Role: LEADER
I20260809 02:14:06.178990 27791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:06.179090 27796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f7393d874645bcadd566c3732b8622" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44777 } }
I20260809 02:14:06.179553 27797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89f7393d874645bcadd566c3732b8622" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f7393d874645bcadd566c3732b8622" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44777 } } }
I20260809 02:14:06.179579 27798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89f7393d874645bcadd566c3732b8622. Latest consensus state: current_term: 1 leader_uuid: "89f7393d874645bcadd566c3732b8622" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f7393d874645bcadd566c3732b8622" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44777 } } }
I20260809 02:14:06.179670 27797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:06.179682 27798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:06.179904 27800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:06.180064 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 27717
I20260809 02:14:06.180198 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:14:06.180393 5158 external_mini_cluster.cc:1428] Setting key 61a261cd9cd6984655cf0203780f5396
I20260809 02:14:06.180433 27800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:06.182217 27800 catalog_manager.cc:1347] Generated new cluster ID: f2b52810f3ea47e1bae475728557fbdf
I20260809 02:14:06.182293 27800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:06.186442 27816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:06.180947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:57409 (local address 127.0.0.1:44777)
0809 02:14:06.181124 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:06.181133 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:06.181349 (+ 216us) server_negotiation.cc:407] Connection header received
0809 02:14:06.181476 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:06.181483 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:06.181518 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:06.181715 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:06.185933 (+ 4218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:06.185937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:06.185938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:06.185943 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:06.185998 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:06.186011 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:06.186096 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:06.186175 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 02:14:06.186231 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":73,"threads_started":1}
I20260809 02:14:06.187631 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:06.190773 27816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:06.189642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:60661 (local address 127.0.0.1:44777)
0809 02:14:06.189681 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:06.189683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:06.189691 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:14:06.189712 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:06.189713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:06.189718 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:06.189772 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:06.190530 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:06.190531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:06.190531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:06.190535 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:06.190553 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:06.190558 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:06.190606 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:06.190679 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 02:14:06.190696 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 02:14:06.192660 27800 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:06.192836 27800 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:06.210626 27800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89f7393d874645bcadd566c3732b8622: Generated new TSK 0
I20260809 02:14:06.210882 27800 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:14:06.218323 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 27717
2026-08-09T02:14:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1378 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:06 dist-test-slave-p3t5 krb5kdc[27826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:06 dist-test-slave-p3t5 krb5kdc[27826](info): set up 2 sockets
Aug 09 02:14:06 dist-test-slave-p3t5 krb5kdc[27826](info): commencing operation
krb5kdc: starting...
W20260809 02:14:07.298646 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:14:07 dist-test-slave-p3t5 krb5kdc[27826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:14:07.442435 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:41573
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:42277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:41573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:07.528540 27839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:07.528826 27839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:07.528869 27839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:07.530140 27839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:07.530216 27839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:07.530241 27839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:07.530263 27839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:07.530292 27839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:07.531822 27839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:42277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:41573
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:41573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:07.532117 27839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:07.532366 27839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:07.537708 27845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:07.537784 27839 server_base.cc:1034] running on GCE node
W20260809 02:14:07.537725 27847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:07.537746 27844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:07.538385 27839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:07.538710 27839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:07.539912 27839 hybrid_clock.cc:648] HybridClock initialized: now 1786241647539894 us; error 29 us; skew 500 ppm
Aug 09 02:14:07 dist-test-slave-p3t5 krb5kdc[27826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241647, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:07.549063 27839 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:14:07.549683 27839 webserver.cc:460] Webserver started at http://127.5.9.190:45901/ using document root <none> and password file <none>
I20260809 02:14:07.549968 27839 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:07.550050 27839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:07.550226 27839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:07.551409 27839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "bb704eca338946569dd5d0d4f4317f5c"
format_stamp: "Formatted at 2026-08-09 02:14:07 on dist-test-slave-p3t5"
server_key: "fa7d9ff3891569857590033bd8f5f1ef"
server_key_iv: "13bf0a3a1707edcbc3ee91ea76a669b0"
server_key_version: "encryptionkey@0"
I20260809 02:14:07.551837 27839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "bb704eca338946569dd5d0d4f4317f5c"
format_stamp: "Formatted at 2026-08-09 02:14:07 on dist-test-slave-p3t5"
server_key: "fa7d9ff3891569857590033bd8f5f1ef"
server_key_iv: "13bf0a3a1707edcbc3ee91ea76a669b0"
server_key_version: "encryptionkey@0"
I20260809 02:14:07.553354 27839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:07.554347 27854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:07.554574 27839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:07.554690 27839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "bb704eca338946569dd5d0d4f4317f5c"
format_stamp: "Formatted at 2026-08-09 02:14:07 on dist-test-slave-p3t5"
server_key: "fa7d9ff3891569857590033bd8f5f1ef"
server_key_iv: "13bf0a3a1707edcbc3ee91ea76a669b0"
server_key_version: "encryptionkey@0"
I20260809 02:14:07.554822 27839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:07.579597 27839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:07.582100 27839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:07.582307 27839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:07.586872 27839 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:41573
I20260809 02:14:07.586938 27910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:41573 every 8 connection(s)
I20260809 02:14:07.587352 27839 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:07.588267 27911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:07.588990 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 27839
I20260809 02:14:07.589155 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:14:07.589367 5158 external_mini_cluster.cc:1428] Setting key d057b5d9a33f43af5fba2911f2dfdbc5
I20260809 02:14:07.591765 27911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c: Bootstrap starting.
I20260809 02:14:07.592444 27911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:07.592782 27911 log.cc:824] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:07.593719 27911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c: No bootstrap required, opened a new log
I20260809 02:14:07.596537 27911 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb704eca338946569dd5d0d4f4317f5c" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41573 } }
I20260809 02:14:07.596695 27911 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:07.596719 27911 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb704eca338946569dd5d0d4f4317f5c, State: Initialized, Role: FOLLOWER
I20260809 02:14:07.596865 27911 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb704eca338946569dd5d0d4f4317f5c" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41573 } }
I20260809 02:14:07.596951 27911 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:07.596974 27911 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:07.597038 27911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [term 0 FOLLOWER]: Advancing to term 1
Aug 09 02:14:07 dist-test-slave-p3t5 krb5kdc[27826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:14:07.597837 27911 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb704eca338946569dd5d0d4f4317f5c" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41573 } }
I20260809 02:14:07.597967 27911 leader_election.cc:302] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb704eca338946569dd5d0d4f4317f5c; no voters:
I20260809 02:14:07.598199 27911 leader_election.cc:288] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:07.598232 27916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:07.598354 27916 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [term 1 LEADER]: Becoming Leader. State: Replica: bb704eca338946569dd5d0d4f4317f5c, State: Running, Role: LEADER
I20260809 02:14:07.598518 27911 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:07.598508 27916 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb704eca338946569dd5d0d4f4317f5c" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41573 } }
I20260809 02:14:07.599151 27917 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb704eca338946569dd5d0d4f4317f5c. Latest consensus state: current_term: 1 leader_uuid: "bb704eca338946569dd5d0d4f4317f5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb704eca338946569dd5d0d4f4317f5c" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41573 } } }
I20260809 02:14:07.599244 27917 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:07.599498 27918 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb704eca338946569dd5d0d4f4317f5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb704eca338946569dd5d0d4f4317f5c" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 41573 } } }
I20260809 02:14:07.599663 27918 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:07.599943 27922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:07.600605 27922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:07.602509 27922 catalog_manager.cc:1347] Generated new cluster ID: cc19e752fd9d431f8e7aa3f2aab3d446
I20260809 02:14:07.602567 27922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:07.606264 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:07.590010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60383 (local address 127.5.9.190:41573)
0809 02:14:07.590218 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:07.590228 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:07.590265 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:14:07.590698 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:07.590704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:07.590762 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:07.590883 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:07.591803 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:07.592935 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:07.593993 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:07.594423 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:07.594515 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:07.597910 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:07.597915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:07.597915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:07.597931 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:07.600973 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:07.601544 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:07.601548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:07.601548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:07.601684 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:07.601988 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:07.601989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:07.601990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:07.602324 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:07.605878 (+ 3554us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:07.605978 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 02:14:07.606087 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":112,"threads_started":1}
I20260809 02:14:07.607772 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:07.616281 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:07.609343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57265 (local address 127.5.9.190:41573)
0809 02:14:07.609385 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:07.609388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:07.609505 (+ 117us) server_negotiation.cc:407] Connection header received
0809 02:14:07.609551 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:07.609551 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:07.609562 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:07.609647 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:07.610252 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:07.610775 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:07.611544 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:07.611790 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:07.611880 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:07.613277 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:07.613278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:07.613279 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:07.613286 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:07.614932 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:07.615359 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:07.615361 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:07.615361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:07.615430 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:07.615689 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:07.615690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:07.615690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:07.615822 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:07.615968 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:07.616094 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 02:14:07.616161 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:14:07.622874 27922 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:07.623646 27922 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:07.627804 27922 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb704eca338946569dd5d0d4f4317f5c: Generated new TSK 0
I20260809 02:14:07.628023 27922 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:14:07.635046 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 27839
W20260809 02:14:07.643239 27824 connection.cc:535] client connection to 127.5.9.190:41573 recv error: Network error: failed to read from TLS socket (remote: 127.5.9.190:41573): Connection reset by peer (error 104)
2026-08-09T02:14:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1418 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:07 dist-test-slave-p3t5 krb5kdc[27944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:07 dist-test-slave-p3t5 krb5kdc[27944](info): set up 2 sockets
Aug 09 02:14:07 dist-test-slave-p3t5 krb5kdc[27944](info): commencing operation
krb5kdc: starting...
W20260809 02:14:08.718112 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:14:08 dist-test-slave-p3t5 krb5kdc[27944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:14:08.861537 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:35171
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:35171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:08.953891 27958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:08.954124 27958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:08.954149 27958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:08.955325 27958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:08.955371 27958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:08.955385 27958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:08.955399 27958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:08.955411 27958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:08.957410 27958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:35171
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:35171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:08.957763 27958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:08.958038 27958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:08.963409 27963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:08.963409 27964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:08.963426 27966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:08.963726 27958 server_base.cc:1034] running on GCE node
I20260809 02:14:08.964005 27958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:08.964293 27958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:08.965449 27958 hybrid_clock.cc:648] HybridClock initialized: now 1786241648965417 us; error 44 us; skew 500 ppm
Aug 09 02:14:08 dist-test-slave-p3t5 krb5kdc[27944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241648, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:08.968271 27958 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:14:08.968847 27958 webserver.cc:460] Webserver started at http://127.5.9.190:39843/ using document root <none> and password file <none>
I20260809 02:14:08.969123 27958 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:08.969198 27958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:08.969370 27958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:08.970610 27958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "76c3de1e770441659091503b85470b32"
format_stamp: "Formatted at 2026-08-09 02:14:08 on dist-test-slave-p3t5"
server_key: "14cecd14c58981dad343c7b06d011777"
server_key_iv: "c8f5aee3d277977a46c973d61b0ff3f0"
server_key_version: "encryptionkey@0"
I20260809 02:14:08.971021 27958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "76c3de1e770441659091503b85470b32"
format_stamp: "Formatted at 2026-08-09 02:14:08 on dist-test-slave-p3t5"
server_key: "14cecd14c58981dad343c7b06d011777"
server_key_iv: "c8f5aee3d277977a46c973d61b0ff3f0"
server_key_version: "encryptionkey@0"
I20260809 02:14:08.972398 27958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:14:08.973224 27973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:08.973448 27958 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:08.973558 27958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "76c3de1e770441659091503b85470b32"
format_stamp: "Formatted at 2026-08-09 02:14:08 on dist-test-slave-p3t5"
server_key: "14cecd14c58981dad343c7b06d011777"
server_key_iv: "c8f5aee3d277977a46c973d61b0ff3f0"
server_key_version: "encryptionkey@0"
I20260809 02:14:08.973667 27958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:09.002377 27958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:09.004897 27958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:09.005088 27958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:09.010257 27958 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:35171
I20260809 02:14:09.010295 28027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:35171 every 8 connection(s)
I20260809 02:14:09.010717 27958 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:09.011691 28028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:09.014159 28028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32: Bootstrap starting.
I20260809 02:14:09.014806 28028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:09.015125 28028 log.cc:824] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:09.015985 28028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32: No bootstrap required, opened a new log
I20260809 02:14:09.017721 28028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c3de1e770441659091503b85470b32" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 35171 } }
I20260809 02:14:09.017886 28028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:09.017910 28028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c3de1e770441659091503b85470b32, State: Initialized, Role: FOLLOWER
I20260809 02:14:09.018050 28028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c3de1e770441659091503b85470b32" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 35171 } }
I20260809 02:14:09.018134 28028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:09.018159 28028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:09.018229 28028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:09.018592 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 27958
I20260809 02:14:09.018745 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:14:09.018993 5158 external_mini_cluster.cc:1428] Setting key 3ee4e73eefa3abf0f969ed9a472b3d5d
I20260809 02:14:09.019397 28028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c3de1e770441659091503b85470b32" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 35171 } }
I20260809 02:14:09.019585 28028 leader_election.cc:302] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76c3de1e770441659091503b85470b32; no voters:
I20260809 02:14:09.019779 28028 leader_election.cc:288] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:09.019822 28033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:09.020052 28033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [term 1 LEADER]: Becoming Leader. State: Replica: 76c3de1e770441659091503b85470b32, State: Running, Role: LEADER
I20260809 02:14:09.020085 28028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:09.020211 28033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c3de1e770441659091503b85470b32" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 35171 } }
I20260809 02:14:09.020680 28033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76c3de1e770441659091503b85470b32. Latest consensus state: current_term: 1 leader_uuid: "76c3de1e770441659091503b85470b32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c3de1e770441659091503b85470b32" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 35171 } } }
I20260809 02:14:09.020779 28033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:09.020692 28035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76c3de1e770441659091503b85470b32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c3de1e770441659091503b85470b32" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 35171 } } }
I20260809 02:14:09.020979 28035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:09.021762 28041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:09.022487 28041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:09.024258 28041 catalog_manager.cc:1347] Generated new cluster ID: 21ed367823f44af790c56e8e71091a9a
I20260809 02:14:09.024313 28041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:09.027046 28032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:09.019566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46825 (local address 127.5.9.190:35171)
0809 02:14:09.019823 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:09.019834 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:09.019872 (+ 38us) server_negotiation.cc:407] Connection header received
0809 02:14:09.020052 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:09.020060 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:09.020129 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:09.020319 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:09.020608 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:09.022179 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:09.023292 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:09.023890 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:09.024019 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:09.026244 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:09.026249 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:09.026249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:09.026493 (+ 244us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:09.026536 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:09.026552 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:09.026632 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:09.026786 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 02:14:09.026817 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":197,"thread_start_us":127,"threads_started":1}
I20260809 02:14:09.028096 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:09.029896 28041 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:09.030652 28041 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:09.034016 28032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:09.029950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49569 (local address 127.5.9.190:35171)
0809 02:14:09.029985 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:09.029987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:09.030237 (+ 250us) server_negotiation.cc:407] Connection header received
0809 02:14:09.030287 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:09.030288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:09.030299 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:09.030370 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:09.030550 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:09.031317 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:09.032493 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:09.032798 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:09.032877 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:09.033734 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:09.033735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:09.033736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:09.033740 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:09.033755 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:09.033760 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:09.033790 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:09.033915 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 02:14:09.033930 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 02:14:09.040074 28041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76c3de1e770441659091503b85470b32: Generated new TSK 0
I20260809 02:14:09.040315 28041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:14:09.046113 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 27958
W20260809 02:14:09.051982 27942 connection.cc:535] client connection to 127.5.9.190:35171 recv error: Network error: failed to read from TLS socket (remote: 127.5.9.190:35171): Connection reset by peer (error 104)
2026-08-09T02:14:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1408 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:09 dist-test-slave-p3t5 krb5kdc[28062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:09 dist-test-slave-p3t5 krb5kdc[28062](info): set up 2 sockets
Aug 09 02:14:09 dist-test-slave-p3t5 krb5kdc[28062](info): commencing operation
krb5kdc: starting...
W20260809 02:14:10.150142 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:14:10 dist-test-slave-p3t5 krb5kdc[28062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:14:10.287122 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:43623
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:43623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:10.373796 28075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:10.374033 28075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:10.374059 28075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:10.375277 28075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:10.375329 28075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:10.375344 28075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:10.375356 28075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:10.375413 28075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:10.377009 28075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:43623
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:43623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:10.377292 28075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:10.377559 28075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:10.382709 28084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:10.382728 28082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:10.382699 28081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:10.382965 28075 server_base.cc:1034] running on GCE node
I20260809 02:14:10.383216 28075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:10.383510 28075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:10.384668 28075 hybrid_clock.cc:648] HybridClock initialized: now 1786241650384657 us; error 32 us; skew 500 ppm
Aug 09 02:14:10 dist-test-slave-p3t5 krb5kdc[28062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241650, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:10.387535 28075 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:14:10.388056 28075 webserver.cc:460] Webserver started at http://127.5.9.190:45975/ using document root <none> and password file <none>
I20260809 02:14:10.388296 28075 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:10.388368 28075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:10.388535 28075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:10.389801 28075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "2059397cabd6420085ee910dfd69da87"
format_stamp: "Formatted at 2026-08-09 02:14:10 on dist-test-slave-p3t5"
server_key: "1d1dda3db4c8fe2ad4478ee6895fa6f1"
server_key_iv: "5439f9a3a45ce0e91c08868123126cc1"
server_key_version: "encryptionkey@0"
I20260809 02:14:10.390203 28075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "2059397cabd6420085ee910dfd69da87"
format_stamp: "Formatted at 2026-08-09 02:14:10 on dist-test-slave-p3t5"
server_key: "1d1dda3db4c8fe2ad4478ee6895fa6f1"
server_key_iv: "5439f9a3a45ce0e91c08868123126cc1"
server_key_version: "encryptionkey@0"
I20260809 02:14:10.391654 28075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:14:10.392489 28091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:10.392683 28075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:10.392786 28075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "2059397cabd6420085ee910dfd69da87"
format_stamp: "Formatted at 2026-08-09 02:14:10 on dist-test-slave-p3t5"
server_key: "1d1dda3db4c8fe2ad4478ee6895fa6f1"
server_key_iv: "5439f9a3a45ce0e91c08868123126cc1"
server_key_version: "encryptionkey@0"
I20260809 02:14:10.392871 28075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:10.396394 28075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:10.398836 28075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:10.399011 28075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:10.403429 28075 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:43623
I20260809 02:14:10.403474 28145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:43623 every 8 connection(s)
I20260809 02:14:10.403856 28075 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:10.404963 28146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:10.407563 28146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87: Bootstrap starting.
I20260809 02:14:10.408154 28146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:10.408403 28146 log.cc:824] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:10.409219 28146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87: No bootstrap required, opened a new log
I20260809 02:14:10.410950 28146 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2059397cabd6420085ee910dfd69da87" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43623 } }
I20260809 02:14:10.411099 28146 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:10.411123 28146 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2059397cabd6420085ee910dfd69da87, State: Initialized, Role: FOLLOWER
I20260809 02:14:10.411232 28146 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2059397cabd6420085ee910dfd69da87" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43623 } }
I20260809 02:14:10.411325 28146 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:10.411365 28146 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:10.411445 28146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:10.412405 28146 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2059397cabd6420085ee910dfd69da87" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43623 } }
I20260809 02:14:10.412541 28146 leader_election.cc:302] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2059397cabd6420085ee910dfd69da87; no voters:
I20260809 02:14:10.412734 28146 leader_election.cc:288] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:10.412884 28150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:10.413031 28146 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:10.413113 28150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [term 1 LEADER]: Becoming Leader. State: Replica: 2059397cabd6420085ee910dfd69da87, State: Running, Role: LEADER
I20260809 02:14:10.413223 28150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2059397cabd6420085ee910dfd69da87" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43623 } }
I20260809 02:14:10.413523 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 28075
I20260809 02:14:10.413693 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:14:10.413748 28151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2059397cabd6420085ee910dfd69da87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2059397cabd6420085ee910dfd69da87" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43623 } } }
I20260809 02:14:10.413829 28151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:10.413918 5158 external_mini_cluster.cc:1428] Setting key 3737f0179ee2d400fe6da4cca3758cdb
I20260809 02:14:10.414011 28152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2059397cabd6420085ee910dfd69da87. Latest consensus state: current_term: 1 leader_uuid: "2059397cabd6420085ee910dfd69da87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2059397cabd6420085ee910dfd69da87" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 43623 } } }
I20260809 02:14:10.414085 28152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:10.414490 28159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:10.415162 28159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:10.417013 28169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:10.414786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56849 (local address 127.5.9.190:43623)
0809 02:14:10.414991 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:10.415002 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:10.415164 (+ 162us) server_negotiation.cc:407] Connection header received
0809 02:14:10.415297 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:10.415304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:10.415338 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:10.415492 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:10.416288 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:10.416292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:10.416292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:10.416298 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:10.416352 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:10.416370 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:10.416648 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:10.416759 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 02:14:10.416806 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260809 02:14:10.417196 28159 catalog_manager.cc:1347] Generated new cluster ID: 0e6702a978dd4807ae95a455603e56cd
I20260809 02:14:10.417309 28159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:10.418265 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:10.421020 28169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:10.419768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59055 (local address 127.5.9.190:43623)
0809 02:14:10.419809 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:10.419812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:10.419929 (+ 117us) server_negotiation.cc:407] Connection header received
0809 02:14:10.420011 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:10.420012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:10.420018 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:10.420080 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:10.420761 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:10.420763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:10.420763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:10.420769 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:10.420784 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:10.420789 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:10.420846 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:10.420906 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 02:14:10.420923 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:14:10.456578 28159 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:10.456799 28159 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:10.463115 28159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2059397cabd6420085ee910dfd69da87: Generated new TSK 0
I20260809 02:14:10.463353 28159 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:14:10.472425 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 28075
2026-08-09T02:14:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:10 dist-test-slave-p3t5 krb5kdc[28178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:10 dist-test-slave-p3t5 krb5kdc[28178](info): set up 2 sockets
Aug 09 02:14:10 dist-test-slave-p3t5 krb5kdc[28178](info): commencing operation
krb5kdc: starting...
W20260809 02:14:11.582633 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:14:11 dist-test-slave-p3t5 krb5kdc[28178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:14:11.716993 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:38197
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:37057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:38197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:11.804430 28191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:11.804683 28191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:11.804711 28191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:11.806094 28191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:11.806149 28191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:11.806166 28191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:11.806180 28191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:11.806247 28191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:11.807997 28191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:37057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:38197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:38197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:11.808274 28191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:11.808561 28191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:11.814138 28197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:11.814225 28199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:11.814150 28196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:11.814424 28191 server_base.cc:1034] running on GCE node
I20260809 02:14:11.814711 28191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:11.814982 28191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:11.816130 28191 hybrid_clock.cc:648] HybridClock initialized: now 1786241651816120 us; error 43 us; skew 500 ppm
Aug 09 02:14:11 dist-test-slave-p3t5 krb5kdc[28178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241651, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:11.818879 28191 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:14:11.819365 28191 webserver.cc:460] Webserver started at http://127.5.9.190:37879/ using document root <none> and password file <none>
I20260809 02:14:11.819636 28191 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:11.819715 28191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:11.819871 28191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:11.821000 28191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "c320e1c5f0f64e10ba091d24c28dfb62"
format_stamp: "Formatted at 2026-08-09 02:14:11 on dist-test-slave-p3t5"
server_key: "b85691fe5f333598b9eabaaf503ab40b"
server_key_iv: "3ac98b3a458d633df18b38ace9a1b605"
server_key_version: "encryptionkey@0"
I20260809 02:14:11.821393 28191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "c320e1c5f0f64e10ba091d24c28dfb62"
format_stamp: "Formatted at 2026-08-09 02:14:11 on dist-test-slave-p3t5"
server_key: "b85691fe5f333598b9eabaaf503ab40b"
server_key_iv: "3ac98b3a458d633df18b38ace9a1b605"
server_key_version: "encryptionkey@0"
I20260809 02:14:11.822974 28191 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:11.823845 28207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:11.824076 28191 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:11.824195 28191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "c320e1c5f0f64e10ba091d24c28dfb62"
format_stamp: "Formatted at 2026-08-09 02:14:11 on dist-test-slave-p3t5"
server_key: "b85691fe5f333598b9eabaaf503ab40b"
server_key_iv: "3ac98b3a458d633df18b38ace9a1b605"
server_key_version: "encryptionkey@0"
I20260809 02:14:11.824280 28191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:11.839493 28191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:11.842162 28191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:11.842351 28191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:11.846680 28191 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:38197
I20260809 02:14:11.846740 28261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:38197 every 8 connection(s)
I20260809 02:14:11.847182 28191 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:11.848152 28262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:11.850615 28262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62: Bootstrap starting.
I20260809 02:14:11.851292 28262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:11.851612 28262 log.cc:824] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:11.852392 28262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62: No bootstrap required, opened a new log
I20260809 02:14:11.853296 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 28191
I20260809 02:14:11.853394 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:14:11.853600 5158 external_mini_cluster.cc:1428] Setting key 927cbbd475191fb293c090857a109e21
I20260809 02:14:11.854635 28262 raft_consensus.cc:348] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c320e1c5f0f64e10ba091d24c28dfb62" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 38197 } }
I20260809 02:14:11.854817 28262 raft_consensus.cc:374] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:11.854857 28262 raft_consensus.cc:729] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c320e1c5f0f64e10ba091d24c28dfb62, State: Initialized, Role: FOLLOWER
I20260809 02:14:11.855055 28262 consensus_queue.cc:260] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c320e1c5f0f64e10ba091d24c28dfb62" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 38197 } }
I20260809 02:14:11.855130 28262 raft_consensus.cc:388] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:11.855152 28262 raft_consensus.cc:482] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:11.855223 28262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:11.856388 28262 raft_consensus.cc:504] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c320e1c5f0f64e10ba091d24c28dfb62" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 38197 } }
I20260809 02:14:11.856515 28262 leader_election.cc:302] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c320e1c5f0f64e10ba091d24c28dfb62; no voters:
I20260809 02:14:11.856698 28262 leader_election.cc:288] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:11.856762 28267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:11.856941 28262 sys_catalog.cc:564] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:11.856945 28267 raft_consensus.cc:686] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [term 1 LEADER]: Becoming Leader. State: Replica: c320e1c5f0f64e10ba091d24c28dfb62, State: Running, Role: LEADER
I20260809 02:14:11.857173 28267 consensus_queue.cc:237] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c320e1c5f0f64e10ba091d24c28dfb62" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 38197 } }
I20260809 02:14:11.857649 28268 sys_catalog.cc:455] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c320e1c5f0f64e10ba091d24c28dfb62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c320e1c5f0f64e10ba091d24c28dfb62" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 38197 } } }
I20260809 02:14:11.857733 28268 sys_catalog.cc:458] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:11.857963 28269 sys_catalog.cc:455] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c320e1c5f0f64e10ba091d24c28dfb62. Latest consensus state: current_term: 1 leader_uuid: "c320e1c5f0f64e10ba091d24c28dfb62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c320e1c5f0f64e10ba091d24c28dfb62" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 38197 } } }
I20260809 02:14:11.858070 28269 sys_catalog.cc:458] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:11.858328 28276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:11.858971 28276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:11.860723 28276 catalog_manager.cc:1347] Generated new cluster ID: b894ae657d17407887fc2c8844b8017b
I20260809 02:14:11.860776 28276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:14:11 dist-test-slave-p3t5 krb5kdc[28178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:14:11.866660 28266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:11.854226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46309 (local address 127.5.9.190:38197)
0809 02:14:11.854456 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:11.854467 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:11.854507 (+ 40us) server_negotiation.cc:407] Connection header received
0809 02:14:11.854634 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:11.854642 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:11.854701 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:11.854858 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:11.855726 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:11.857176 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:11.858314 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:11.858928 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:11.859060 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:11.862115 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:11.862120 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:11.862120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:11.862134 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:11.864232 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:11.864665 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:11.864668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:11.864669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:11.864809 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:11.865048 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:11.865049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:11.865050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:11.865396 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:11.865572 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:11.866199 (+ 627us) server_negotiation.cc:299] Negotiation successful
0809 02:14:11.866398 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":102,"threads_started":1}
I20260809 02:14:11.867874 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:11.871255 28276 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:11.871925 28276 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:11.875607 28276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c320e1c5f0f64e10ba091d24c28dfb62: Generated new TSK 0
I20260809 02:14:11.875815 28276 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:14:11.877017 28266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:11.869388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60673 (local address 127.5.9.190:38197)
0809 02:14:11.869431 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:11.869433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:11.869651 (+ 218us) server_negotiation.cc:407] Connection header received
0809 02:14:11.869710 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:11.869711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:11.869736 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:11.869803 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:11.870467 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:11.871136 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:11.872144 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:11.872486 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:11.872563 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:11.874078 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:11.874080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:11.874081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:11.874088 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:11.875795 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:11.876212 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:11.876213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:11.876213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:11.876282 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:11.876483 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:11.876484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:11.876484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:11.876625 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:11.876734 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:11.876806 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 02:14:11.876909 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:14:11.879395 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 28191
2026-08-09T02:14:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:11 dist-test-slave-p3t5 krb5kdc[28295](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:11 dist-test-slave-p3t5 krb5kdc[28295](info): set up 2 sockets
Aug 09 02:14:11 dist-test-slave-p3t5 krb5kdc[28295](info): commencing operation
krb5kdc: starting...
W20260809 02:14:12.982583 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:14:13 dist-test-slave-p3t5 krb5kdc[28295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:14:13.122155 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:45221
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:33787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:45221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:13.210172 28308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:13.210450 28308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:13.210494 28308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:13.211685 28308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:13.211772 28308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:13.211797 28308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:13.211827 28308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:13.211860 28308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:13.213410 28308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:33787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:45221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:45221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:13.213728 28308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:13.213986 28308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:14:13.219497 28308 server_base.cc:1034] running on GCE node
W20260809 02:14:13.219457 28315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:13.219444 28314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:13.219446 28317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:13.219944 28308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:13.220314 28308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:13.221500 28308 hybrid_clock.cc:648] HybridClock initialized: now 1786241653221486 us; error 51 us; skew 500 ppm
Aug 09 02:14:13 dist-test-slave-p3t5 krb5kdc[28295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241653, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:13.230459 28308 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:14:13.231042 28308 webserver.cc:460] Webserver started at http://127.5.9.190:34107/ using document root <none> and password file <none>
I20260809 02:14:13.231266 28308 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:13.231312 28308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:13.231418 28308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:13.232631 28308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "97e92879b4874442a8bebd1d986893f2"
format_stamp: "Formatted at 2026-08-09 02:14:13 on dist-test-slave-p3t5"
server_key: "9d7fa2e4f5a68af1bab55c3ba7cb561d"
server_key_iv: "15d895c1e861bde13c407af0da8eb0f0"
server_key_version: "encryptionkey@0"
I20260809 02:14:13.233006 28308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "97e92879b4874442a8bebd1d986893f2"
format_stamp: "Formatted at 2026-08-09 02:14:13 on dist-test-slave-p3t5"
server_key: "9d7fa2e4f5a68af1bab55c3ba7cb561d"
server_key_iv: "15d895c1e861bde13c407af0da8eb0f0"
server_key_version: "encryptionkey@0"
I20260809 02:14:13.234607 28308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260809 02:14:13.235518 28324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:13.235828 28308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:13.235950 28308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "97e92879b4874442a8bebd1d986893f2"
format_stamp: "Formatted at 2026-08-09 02:14:13 on dist-test-slave-p3t5"
server_key: "9d7fa2e4f5a68af1bab55c3ba7cb561d"
server_key_iv: "15d895c1e861bde13c407af0da8eb0f0"
server_key_version: "encryptionkey@0"
I20260809 02:14:13.236106 28308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:13.251652 28308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:13.253808 28308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:13.253984 28308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:13.258529 28308 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:45221
I20260809 02:14:13.258592 28378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:45221 every 8 connection(s)
I20260809 02:14:13.258988 28308 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:13.259909 28379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:13.262514 28379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2: Bootstrap starting.
I20260809 02:14:13.263185 28379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:13.263501 28379 log.cc:824] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:13.264348 28379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2: No bootstrap required, opened a new log
I20260809 02:14:13.266150 28379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e92879b4874442a8bebd1d986893f2" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45221 } }
I20260809 02:14:13.266297 28379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:13.266321 28379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e92879b4874442a8bebd1d986893f2, State: Initialized, Role: FOLLOWER
I20260809 02:14:13.266490 28379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e92879b4874442a8bebd1d986893f2" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45221 } }
I20260809 02:14:13.266583 28379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:13.266620 28379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:13.266700 28379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:13.267645 28379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e92879b4874442a8bebd1d986893f2" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45221 } }
I20260809 02:14:13.267779 28379 leader_election.cc:302] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97e92879b4874442a8bebd1d986893f2; no voters:
I20260809 02:14:13.267987 28379 leader_election.cc:288] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:13.268093 28382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:13.268283 28379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:13.268286 28382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [term 1 LEADER]: Becoming Leader. State: Replica: 97e92879b4874442a8bebd1d986893f2, State: Running, Role: LEADER
I20260809 02:14:13.268606 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 28308
I20260809 02:14:13.268621 28382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e92879b4874442a8bebd1d986893f2" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45221 } }
I20260809 02:14:13.268728 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:14:13.268944 5158 external_mini_cluster.cc:1428] Setting key b75588cedf8ca0db909f76118de17c37
I20260809 02:14:13.269173 28384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97e92879b4874442a8bebd1d986893f2. Latest consensus state: current_term: 1 leader_uuid: "97e92879b4874442a8bebd1d986893f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e92879b4874442a8bebd1d986893f2" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45221 } } }
I20260809 02:14:13.269285 28384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:13.269385 28383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97e92879b4874442a8bebd1d986893f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e92879b4874442a8bebd1d986893f2" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 45221 } } }
I20260809 02:14:13.269495 28383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2 [sys.catalog]: This master's current role is: LEADER
W20260809 02:14:13.270342 28399 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:14:13.270417 28399 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:14:13.270736 28394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:13.270942 28394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:13.272850 28394 catalog_manager.cc:1347] Generated new cluster ID: ab292b4abe9647758ca07445cbd32fa6
I20260809 02:14:13.272946 28394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:13.275713 28400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:13.269683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43481 (local address 127.5.9.190:45221)
0809 02:14:13.269878 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:13.269889 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:13.270272 (+ 383us) server_negotiation.cc:407] Connection header received
0809 02:14:13.270414 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:13.270422 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:13.270484 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:13.270629 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:13.270940 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:13.272192 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:13.273392 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:13.274034 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:13.274141 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:13.274951 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:13.274956 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:13.274956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:13.275172 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:13.275209 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:13.275224 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:13.275287 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:13.275413 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 02:14:13.275483 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":74,"threads_started":1}
I20260809 02:14:13.276880 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:13.282409 28400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:13.278712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55181 (local address 127.5.9.190:45221)
0809 02:14:13.278746 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:13.278748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:13.278851 (+ 103us) server_negotiation.cc:407] Connection header received
0809 02:14:13.278913 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:13.278914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:13.278926 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:13.278984 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:13.279133 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:13.279858 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:13.280939 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:13.281246 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:13.281311 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:13.282130 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:13.282132 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:13.282132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:13.282137 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:13.282154 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:13.282159 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:13.282215 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:13.282295 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 02:14:13.282322 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 02:14:13.283535 28394 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:13.284103 28394 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:13.288203 28394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97e92879b4874442a8bebd1d986893f2: Generated new TSK 0
I20260809 02:14:13.288419 28394 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:14:13.291116 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 28308
2026-08-09T02:14:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:13 dist-test-slave-p3t5 krb5kdc[28412](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:13 dist-test-slave-p3t5 krb5kdc[28412](info): set up 2 sockets
Aug 09 02:14:13 dist-test-slave-p3t5 krb5kdc[28412](info): commencing operation
krb5kdc: starting...
W20260809 02:14:14.380242 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:14:14 dist-test-slave-p3t5 krb5kdc[28412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:14:14.517264 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:33931
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:43995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:33931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:14.608659 28426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:14.608939 28426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:14.608982 28426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:14.610294 28426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:14.610368 28426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:14.610394 28426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:14.610423 28426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:14.610455 28426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:14.612073 28426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:43995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:33931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:33931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:14.612378 28426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:14.612643 28426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:14:14.617995 28426 server_base.cc:1034] running on GCE node
W20260809 02:14:14.617945 28431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:14.617945 28434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:14.617985 28432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:14.618511 28426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:14.618851 28426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:14.620071 28426 hybrid_clock.cc:648] HybridClock initialized: now 1786241654620051 us; error 34 us; skew 500 ppm
Aug 09 02:14:14 dist-test-slave-p3t5 krb5kdc[28412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241654, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:14.622828 28426 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:14:14.623581 28426 webserver.cc:460] Webserver started at http://127.5.9.190:41507/ using document root <none> and password file <none>
I20260809 02:14:14.623929 28426 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:14.624003 28426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:14.624158 28426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:14.625783 28426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "4e7702776cb3489e848150d3fd208680"
format_stamp: "Formatted at 2026-08-09 02:14:14 on dist-test-slave-p3t5"
server_key: "a6d216ad71b9c30ad80b666fd4b9f68a"
server_key_iv: "36de9aae2b2df8010a2ab3afd38f866c"
server_key_version: "encryptionkey@0"
I20260809 02:14:14.626250 28426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "4e7702776cb3489e848150d3fd208680"
format_stamp: "Formatted at 2026-08-09 02:14:14 on dist-test-slave-p3t5"
server_key: "a6d216ad71b9c30ad80b666fd4b9f68a"
server_key_iv: "36de9aae2b2df8010a2ab3afd38f866c"
server_key_version: "encryptionkey@0"
I20260809 02:14:14.628284 28426 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:14:14.629304 28441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:14.629515 28426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:14.629647 28426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "4e7702776cb3489e848150d3fd208680"
format_stamp: "Formatted at 2026-08-09 02:14:14 on dist-test-slave-p3t5"
server_key: "a6d216ad71b9c30ad80b666fd4b9f68a"
server_key_iv: "36de9aae2b2df8010a2ab3afd38f866c"
server_key_version: "encryptionkey@0"
I20260809 02:14:14.629740 28426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:14.633677 28426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:14.636559 28426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:14.636719 28426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:14.641295 28426 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:33931
I20260809 02:14:14.641330 28497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:33931 every 8 connection(s)
I20260809 02:14:14.641745 28426 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:14.642783 28498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:14.643358 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 28426
I20260809 02:14:14.643489 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:14:14.643700 5158 external_mini_cluster.cc:1428] Setting key 8cf83c875b93e920f2214c45fe93dca0
I20260809 02:14:14.645859 28498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680: Bootstrap starting.
I20260809 02:14:14.646355 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:14.644298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43133 (local address 127.5.9.190:33931)
0809 02:14:14.644501 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:14.644511 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:14.644550 (+ 39us) server_negotiation.cc:407] Connection header received
0809 02:14:14.644964 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:14.644971 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:14.644996 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:14.645157 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:14.645931 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:14.645935 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:14.645936 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:14.645941 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:14.645989 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:14.646007 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:14.646079 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:14.646150 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 02:14:14.646192 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":87,"threads_started":1}
I20260809 02:14:14.646651 28498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:14.647042 28498 log.cc:824] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:14.647444 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:14.647928 28498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680: No bootstrap required, opened a new log
I20260809 02:14:14.650297 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:14.649074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48707 (local address 127.5.9.190:33931)
0809 02:14:14.649114 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:14.649116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:14.649185 (+ 69us) server_negotiation.cc:407] Connection header received
0809 02:14:14.649242 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:14.649242 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:14.649246 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:14.649289 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:14.650028 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:14.650030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:14.650030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:14.650034 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:14.650050 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:14.650054 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:14.650106 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:14.650186 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 02:14:14.650205 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:14:14.650799 28498 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e7702776cb3489e848150d3fd208680" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33931 } }
I20260809 02:14:14.650995 28498 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:14.651042 28498 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e7702776cb3489e848150d3fd208680, State: Initialized, Role: FOLLOWER
I20260809 02:14:14.651170 28498 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e7702776cb3489e848150d3fd208680" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33931 } }
I20260809 02:14:14.651257 28498 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:14.651306 28498 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:14.651357 28498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:14.651953 28498 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e7702776cb3489e848150d3fd208680" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33931 } }
I20260809 02:14:14.652092 28498 leader_election.cc:302] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e7702776cb3489e848150d3fd208680; no voters:
I20260809 02:14:14.652252 28498 leader_election.cc:288] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:14.652283 28508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:14.652462 28508 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [term 1 LEADER]: Becoming Leader. State: Replica: 4e7702776cb3489e848150d3fd208680, State: Running, Role: LEADER
I20260809 02:14:14.652529 28498 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:14.652599 28508 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e7702776cb3489e848150d3fd208680" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33931 } }
I20260809 02:14:14.653097 28510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e7702776cb3489e848150d3fd208680. Latest consensus state: current_term: 1 leader_uuid: "4e7702776cb3489e848150d3fd208680" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e7702776cb3489e848150d3fd208680" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33931 } } }
I20260809 02:14:14.653203 28510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:14.653453 28518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:14.653558 28509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e7702776cb3489e848150d3fd208680" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e7702776cb3489e848150d3fd208680" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 33931 } } }
I20260809 02:14:14.653739 28509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:14.654002 28518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:14.655627 28518 catalog_manager.cc:1347] Generated new cluster ID: 9e18f56467aa48fdbe272310fc874f52
I20260809 02:14:14.655685 28518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:14.662959 28518 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:14.663143 28518 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:14.666028 28518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e7702776cb3489e848150d3fd208680: Generated new TSK 0
I20260809 02:14:14.666220 28518 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:14:14.673219 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 28426
2026-08-09T02:14:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1381 ms)
[----------] 18 tests from AuthTokenIssuingTest (25758 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:14 dist-test-slave-p3t5 krb5kdc[28530](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:14 dist-test-slave-p3t5 krb5kdc[28530](info): set up 2 sockets
Aug 09 02:14:14 dist-test-slave-p3t5 krb5kdc[28530](info): commencing operation
krb5kdc: starting...
W20260809 02:14:15.750063 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:14:15 dist-test-slave-p3t5 krb5kdc[28530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:14:15.884830 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:34305
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:34305,127.5.9.189:39909,127.5.9.188:35629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:15.970124 28545 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:14:15.970422 28545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:15.970458 28545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:15.970505 28545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:15.971675 28545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:15.971761 28545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:15.971788 28545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:15.971822 28545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:15.971845 28545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:15.973398 28545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:34305,127.5.9.189:39909,127.5.9.188:35629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:34305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:15.973727 28545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:15.973990 28545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:15.979328 28550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:15.979328 28553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:15.979346 28551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:15.979579 28545 server_base.cc:1034] running on GCE node
I20260809 02:14:15.979888 28545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:15.980159 28545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:15.981320 28545 hybrid_clock.cc:648] HybridClock initialized: now 1786241655981313 us; error 51 us; skew 500 ppm
Aug 09 02:14:15 dist-test-slave-p3t5 krb5kdc[28530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241655, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:15.984658 28545 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:14:15.985160 28545 webserver.cc:460] Webserver started at http://127.5.9.190:37599/ using document root <none> and password file <none>
I20260809 02:14:15.985437 28545 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:15.985508 28545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:15.985718 28545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:15.986912 28545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "4eedc8ffba634d48b8d05319e59103f0"
format_stamp: "Formatted at 2026-08-09 02:14:15 on dist-test-slave-p3t5"
server_key: "cd857090fb796504792d75261fddbf8c"
server_key_iv: "1336a66b34f3cc447c2c63276eaddf3f"
server_key_version: "encryptionkey@0"
I20260809 02:14:15.987309 28545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "4eedc8ffba634d48b8d05319e59103f0"
format_stamp: "Formatted at 2026-08-09 02:14:15 on dist-test-slave-p3t5"
server_key: "cd857090fb796504792d75261fddbf8c"
server_key_iv: "1336a66b34f3cc447c2c63276eaddf3f"
server_key_version: "encryptionkey@0"
I20260809 02:14:15.988757 28545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:14:15.989585 28561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:15.989851 28545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:15.989950 28545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "4eedc8ffba634d48b8d05319e59103f0"
format_stamp: "Formatted at 2026-08-09 02:14:15 on dist-test-slave-p3t5"
server_key: "cd857090fb796504792d75261fddbf8c"
server_key_iv: "1336a66b34f3cc447c2c63276eaddf3f"
server_key_version: "encryptionkey@0"
I20260809 02:14:15.990038 28545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:15.998888 28545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:16.000814 28545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:16.000979 28545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:16.005082 28545 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:34305
I20260809 02:14:16.005151 28615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:34305 every 8 connection(s)
I20260809 02:14:16.005512 28545 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:16.006618 28616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:16.008276 28616 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:14:16.010890 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 28545
I20260809 02:14:16.011003 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:14:16.011202 5158 external_mini_cluster.cc:1428] Setting key e7af5abad1534f2e53075f0c35f795a6
Aug 09 02:14:16 dist-test-slave-p3t5 krb5kdc[28530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241655, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:14:16.019675 28620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.009100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44058 (local address 127.5.9.190:34305)
0809 02:14:16.009284 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.009288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.009324 (+ 36us) server_negotiation.cc:407] Connection header received
0809 02:14:16.009995 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.009997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.010016 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.010099 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.010733 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.012134 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.013250 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.013573 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.013732 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.016067 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:16.016075 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:16.016075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:16.016083 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:16.017943 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.018378 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.018381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.018382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.018469 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.018712 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.018713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.018713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.018949 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:16.019051 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:16.019240 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 02:14:16.019400 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":95,"threads_started":1}
I20260809 02:14:16.019676 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.009002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:34305 (local address 127.0.0.1:44058)
0809 02:14:16.009191 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.009255 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:16.009805 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:16.010132 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:16.010135 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:16.010419 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:16.010584 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.010588 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.012258 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.012259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.013143 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.013147 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.013728 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.013730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.013826 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.014424 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:16.014433 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:16.015970 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:16.018045 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.018053 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.018055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.018274 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.018552 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.018553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.018553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.018643 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.019027 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:16.019033 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:16.019101 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 02:14:16.019212 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 02:14:16.019469 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":101,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for kudu/127.5.9.189@KRBTEST.COM; defaulting to no policy
I20260809 02:14:16.020474 28616 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:14:16.020897 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.020729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:39909 (local address 127.0.0.1:53902)
0809 02:14:16.020784 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.020872 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 02:14:16.021113 28562 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.9.189:39909: connect: Connection refused (error 111)
I20260809 02:14:16.021517 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.021451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:39909 (local address 127.0.0.1:53914)
0809 02:14:16.021474 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.021499 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260809 02:14:16.021656 28616 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.189:39909: Network error: Client connection negotiation failed: client connection to 127.5.9.189:39909: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.189.keytab.
Entry for principal kudu/127.5.9.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.189.keytab.
I20260809 02:14:16.041438 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.189:39909
--webserver_interface=127.5.9.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.189.keytab
--principal=kudu/127.5.9.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:34305,127.5.9.189:39909,127.5.9.188:35629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:14:16.070914 28616 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } attempt: 1
I20260809 02:14:16.071482 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.071326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:39909 (local address 127.0.0.1:53928)
0809 02:14:16.071404 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.071450 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 02:14:16.071892 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.071779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:39909 (local address 127.0.0.1:53932)
0809 02:14:16.071834 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.071869 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 02:14:16.071986 28616 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.189:39909: Network error: Client connection negotiation failed: client connection to 127.5.9.189:39909: connect: Connection refused (error 111)
W20260809 02:14:16.127602 28625 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:14:16.127884 28625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:16.127918 28625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:16.127961 28625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:16.129168 28625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:16.129245 28625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:16.129269 28625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:16.129290 28625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:16.129323 28625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:16.130908 28625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:34305,127.5.9.189:39909,127.5.9.188:35629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.189.keytab
--principal=kudu/127.5.9.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.189:39909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.9.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:16.131208 28625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:16.131467 28625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:16.136649 28633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:16.136709 28630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:16.136649 28631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:16.136971 28625 server_base.cc:1034] running on GCE node
I20260809 02:14:16.137223 28625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:16.137480 28625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:16.138624 28625 hybrid_clock.cc:648] HybridClock initialized: now 1786241656138607 us; error 35 us; skew 500 ppm
I20260809 02:14:16.140137 28616 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } attempt: 2
I20260809 02:14:16.140623 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.140472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:39909 (local address 127.0.0.1:53944)
0809 02:14:16.140543 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.140582 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 02:14:16.140992 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.140872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:39909 (local address 127.0.0.1:53960)
0809 02:14:16.140945 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.140971 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 02:14:16.141086 28616 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.189:39909: Network error: Client connection negotiation failed: client connection to 127.5.9.189:39909: connect: Connection refused (error 111)
Aug 09 02:14:16 dist-test-slave-p3t5 krb5kdc[28530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241656, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:16.141825 28625 init.cc:377] Logged in from keytab as kudu/127.5.9.189@KRBTEST.COM (short username kudu)
I20260809 02:14:16.142277 28625 webserver.cc:460] Webserver started at http://127.5.9.189:34589/ using document root <none> and password file <none>
I20260809 02:14:16.142515 28625 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:16.142561 28625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:16.142709 28625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:16.143955 28625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/data/instance:
uuid: "4a6c803e9101443ea16a726a6fc4f626"
format_stamp: "Formatted at 2026-08-09 02:14:16 on dist-test-slave-p3t5"
server_key: "66ff77ab9caefb98652149c14c79b775"
server_key_iv: "564714ac13e426c37a7bf9b2cf795a4d"
server_key_version: "encryptionkey@0"
I20260809 02:14:16.144342 28625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/wal/instance:
uuid: "4a6c803e9101443ea16a726a6fc4f626"
format_stamp: "Formatted at 2026-08-09 02:14:16 on dist-test-slave-p3t5"
server_key: "66ff77ab9caefb98652149c14c79b775"
server_key_iv: "564714ac13e426c37a7bf9b2cf795a4d"
server_key_version: "encryptionkey@0"
I20260809 02:14:16.145821 28625 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:16.146611 28640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:16.146862 28625 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:16.146941 28625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/wal
uuid: "4a6c803e9101443ea16a726a6fc4f626"
format_stamp: "Formatted at 2026-08-09 02:14:16 on dist-test-slave-p3t5"
server_key: "66ff77ab9caefb98652149c14c79b775"
server_key_iv: "564714ac13e426c37a7bf9b2cf795a4d"
server_key_version: "encryptionkey@0"
I20260809 02:14:16.147063 28625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:16.170517 28625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:16.172595 28625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:16.172785 28625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:16.177299 28625 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.189:39909
I20260809 02:14:16.177331 28693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.189:39909 every 8 connection(s)
I20260809 02:14:16.177830 28625 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/data/info.pb
I20260809 02:14:16.179024 28694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:16.180641 28694 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:14:16 dist-test-slave-p3t5 krb5kdc[28530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241656, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.189@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:14:16.187894 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 28625
I20260809 02:14:16.188006 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-1/wal/instance
I20260809 02:14:16.188198 5158 external_mini_cluster.cc:1428] Setting key 4cd55d81b684d1b24f0b63eb66539d5f
I20260809 02:14:16.190541 28620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.181458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44070 (local address 127.5.9.190:34305)
0809 02:14:16.181533 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.181535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.181813 (+ 278us) server_negotiation.cc:407] Connection header received
0809 02:14:16.182369 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.182371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.182385 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.182452 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.183132 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.183653 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.184711 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.184948 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.185039 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.187422 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:16.187424 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:16.187424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:16.187431 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:16.189026 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.189471 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.189473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.189473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.189542 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.189838 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.189839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.189839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.189974 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:16.190053 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:16.190350 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 02:14:16.190441 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:14:16.190750 28696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.181420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:34305 (local address 127.0.0.1:44070)
0809 02:14:16.181679 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.181727 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:16.182243 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:16.182528 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:16.182532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:16.182815 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:16.183023 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.183028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.183761 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.183762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.184610 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.184613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.185065 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.185067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.185203 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.185832 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:16.185841 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:16.187324 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:16.189119 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.189128 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.189130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.189374 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.189638 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.189639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.189640 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.189760 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.190045 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:16.190053 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:16.190207 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 02:14:16.190314 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 02:14:16.190496 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":110,"threads_started":1}
I20260809 02:14:16.191321 28694 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:14:16 dist-test-slave-p3t5 krb5kdc[28530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241656, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.189@KRBTEST.COM for kudu/127.5.9.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.9.188@KRBTEST.COM; defaulting to no policy
I20260809 02:14:16.199640 28696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.191659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:39909 (local address 127.0.0.1:53962)
0809 02:14:16.191715 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.191728 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:16.191779 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:16.192041 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:16.192042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:16.192215 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:16.192282 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.192284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.193227 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.193229 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.194170 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.194174 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.194657 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.194658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.194748 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.195232 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:16.195238 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:16.196600 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:16.198484 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.198485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.198486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.198692 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.198927 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.198928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.198928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.198993 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.199289 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:16.199290 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:16.199330 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 02:14:16.199403 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 02:14:16.199514 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 02:14:16.199676 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.191698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53962 (local address 127.5.9.189:39909)
0809 02:14:16.191868 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.191872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.191891 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:14:16.191920 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.191921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.191932 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.192027 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.192357 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.193142 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.194254 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.194575 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.194635 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.196690 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:16.196700 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:16.196700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:16.196716 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:16.198343 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.198791 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.198794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.198795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.198856 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.199060 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.199060 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.199061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.199215 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:16.199312 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:16.199442 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 02:14:16.199568 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":139,"thread_start_us":109,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.188@KRBTEST.COM" created.
I20260809 02:14:16.200425 28694 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:14:16.200817 28696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.200697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:35629 (local address 127.0.0.1:46036)
0809 02:14:16.200747 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.200796 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 02:14:16.200958 28641 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
I20260809 02:14:16.201432 28696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.201332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:35629 (local address 127.0.0.1:46046)
0809 02:14:16.201378 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.201406 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 02:14:16.201538 28694 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:35629: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.188.keytab.
Entry for principal kudu/127.5.9.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.188.keytab.
I20260809 02:14:16.220706 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.188:35629
--webserver_interface=127.5.9.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.188.keytab
--principal=kudu/127.5.9.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:34305,127.5.9.189:39909,127.5.9.188:35629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:14:16.232239 28616 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } attempt: 3
Aug 09 02:14:16 dist-test-slave-p3t5 krb5kdc[28530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241655, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for kudu/127.5.9.189@KRBTEST.COM
I20260809 02:14:16.240069 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.232779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53974 (local address 127.5.9.189:39909)
0809 02:14:16.232844 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.232846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.232855 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:14:16.232900 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.232901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.232911 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.232976 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.233378 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.233966 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.235004 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.235219 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.235275 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.237459 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:16.237461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:16.237461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:16.237469 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:16.238960 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.239345 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.239346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.239346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.239404 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.239599 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.239600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.239600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.239708 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:16.239762 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:16.239927 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 02:14:16.239996 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 02:14:16.240111 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.232702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:39909 (local address 127.0.0.1:53974)
0809 02:14:16.232774 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.232785 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:16.232829 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:16.233010 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:16.233011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:16.233213 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:16.233295 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.233297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.234052 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.234054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.234911 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.234914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.235296 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.235297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.235382 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.235867 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:16.235873 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:16.237364 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:16.239044 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.239045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.239046 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.239270 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.239468 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.239469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.239469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.239538 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.239770 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:16.239771 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:16.239825 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 02:14:16.239895 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 02:14:16.240004 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 02:14:16.240638 28616 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:14:16.241037 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.240915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:35629 (local address 127.0.0.1:46054)
0809 02:14:16.240972 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.241010 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 02:14:16.241376 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.241284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:35629 (local address 127.0.0.1:46056)
0809 02:14:16.241330 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.241358 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 02:14:16.241494 28616 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:35629: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
I20260809 02:14:16.250828 28694 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } attempt: 1
I20260809 02:14:16.251294 28696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.251155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:35629 (local address 127.0.0.1:46058)
0809 02:14:16.251218 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.251260 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 02:14:16.251675 28696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.251563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:35629 (local address 127.0.0.1:46074)
0809 02:14:16.251612 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.251640 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 02:14:16.251765 28694 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:35629: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
I20260809 02:14:16.272697 28616 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } attempt: 1
I20260809 02:14:16.273275 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.273099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:35629 (local address 127.0.0.1:46080)
0809 02:14:16.273173 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.273232 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 02:14:16.273674 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.273569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:35629 (local address 127.0.0.1:46094)
0809 02:14:16.273599 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.273651 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260809 02:14:16.273772 28616 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:35629: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
W20260809 02:14:16.309470 28702 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:14:16.309749 28702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:16.309768 28702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:16.309790 28702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:16.311040 28702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:16.311091 28702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:16.311105 28702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:16.311118 28702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:16.311131 28702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:16.312732 28702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:34305,127.5.9.189:39909,127.5.9.188:35629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.188.keytab
--principal=kudu/127.5.9.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.188:35629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.9.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:16.313040 28702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:16.313318 28702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:16.318583 28707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:16.318616 28711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:16.318588 28708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:16.318743 28702 server_base.cc:1034] running on GCE node
I20260809 02:14:16.319140 28702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:16.319463 28702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:16.319903 28694 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } attempt: 2
I20260809 02:14:16.320293 28696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.320169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:35629 (local address 127.0.0.1:46110)
0809 02:14:16.320231 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.320266 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 02:14:16.320621 28696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.320525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:35629 (local address 127.0.0.1:46118)
0809 02:14:16.320569 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.320593 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 02:14:16.320633 28702 hybrid_clock.cc:648] HybridClock initialized: now 1786241656320620 us; error 48 us; skew 500 ppm
W20260809 02:14:16.320708 28694 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:35629: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
Aug 09 02:14:16 dist-test-slave-p3t5 krb5kdc[28530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241656, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:16.323313 28702 init.cc:377] Logged in from keytab as kudu/127.5.9.188@KRBTEST.COM (short username kudu)
I20260809 02:14:16.323822 28702 webserver.cc:460] Webserver started at http://127.5.9.188:36367/ using document root <none> and password file <none>
I20260809 02:14:16.324066 28702 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:16.324137 28702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:16.324296 28702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:16.325415 28702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/data/instance:
uuid: "56a1130920b749e0b3288fd44a728c70"
format_stamp: "Formatted at 2026-08-09 02:14:16 on dist-test-slave-p3t5"
server_key: "94ba12a2a90a616990127a0e6b0a8ecb"
server_key_iv: "a020cd32b23770c8965a40c33097f40c"
server_key_version: "encryptionkey@0"
I20260809 02:14:16.325829 28702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/wal/instance:
uuid: "56a1130920b749e0b3288fd44a728c70"
format_stamp: "Formatted at 2026-08-09 02:14:16 on dist-test-slave-p3t5"
server_key: "94ba12a2a90a616990127a0e6b0a8ecb"
server_key_iv: "a020cd32b23770c8965a40c33097f40c"
server_key_version: "encryptionkey@0"
I20260809 02:14:16.327248 28702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:16.327967 28719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:16.328143 28702 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:16.328254 28702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/wal
uuid: "56a1130920b749e0b3288fd44a728c70"
format_stamp: "Formatted at 2026-08-09 02:14:16 on dist-test-slave-p3t5"
server_key: "94ba12a2a90a616990127a0e6b0a8ecb"
server_key_iv: "a020cd32b23770c8965a40c33097f40c"
server_key_version: "encryptionkey@0"
I20260809 02:14:16.328331 28702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:16.348958 28616 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } attempt: 2
I20260809 02:14:16.349435 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.349292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:35629 (local address 127.0.0.1:46126)
0809 02:14:16.349362 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.349404 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 02:14:16.349870 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.349730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:35629 (local address 127.0.0.1:46134)
0809 02:14:16.349795 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.349839 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 02:14:16.350003 28616 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:35629: Network error: Client connection negotiation failed: client connection to 127.5.9.188:35629: connect: Connection refused (error 111)
I20260809 02:14:16.350543 28702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:16.352686 28702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:16.352861 28702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:16.357525 28702 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.188:35629
I20260809 02:14:16.357600 28776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.188:35629 every 8 connection(s)
I20260809 02:14:16.357997 28702 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/data/info.pb
I20260809 02:14:16.359036 28777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:16.360692 28777 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:14:16.366969 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 28702
I20260809 02:14:16.367079 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241377393328-5158-0/minicluster-data/master-2/wal/instance
Aug 09 02:14:16 dist-test-slave-p3t5 krb5kdc[28530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241656, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.188@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:14:16.367265 5158 external_mini_cluster.cc:1428] Setting key be90388883204b43ba3850244120a4e1
I20260809 02:14:16.371306 28620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.361656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44086 (local address 127.5.9.190:34305)
0809 02:14:16.361723 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.361725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.361985 (+ 260us) server_negotiation.cc:407] Connection header received
0809 02:14:16.362462 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.362464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.362475 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.362535 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.363210 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.363872 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.364909 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.365114 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.365181 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.367958 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:16.367960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:16.367960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:16.367969 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:16.369762 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.370267 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.370268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.370268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.370329 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.370616 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.370617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.370617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.370727 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:16.370778 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:16.371121 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 02:14:16.371206 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:14:16.371490 28779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.361568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:34305 (local address 127.0.0.1:44086)
0809 02:14:16.361815 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.361857 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:16.362336 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:16.362615 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:16.362618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:16.362899 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:16.363120 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.363125 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.363975 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.363976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.364811 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.364813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.365210 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.365211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.365323 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.365957 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:16.365966 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:16.367749 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:16.369843 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.369850 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.369852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.370183 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.370402 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.370403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.370403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.370548 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.370793 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:16.370799 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:16.370979 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 02:14:16.371076 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 02:14:16.371206 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":148,"threads_started":1}
I20260809 02:14:16.372143 28777 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:14:16 dist-test-slave-p3t5 krb5kdc[28530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.188@KRBTEST.COM
Aug 09 02:14:16 dist-test-slave-p3t5 krb5kdc[28530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241656, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.188@KRBTEST.COM for kudu/127.5.9.189@KRBTEST.COM
I20260809 02:14:16.377128 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.367724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46144 (local address 127.5.9.188:35629)
0809 02:14:16.367901 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.367907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.367969 (+ 62us) server_negotiation.cc:407] Connection header received
0809 02:14:16.368010 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.368013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.368029 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.368121 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.368712 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.369938 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.370934 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.371338 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.371421 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.374015 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:16.374024 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:16.374024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:16.374031 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:16.375642 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.376069 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.376072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.376073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.376161 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.376441 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.376442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.376442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.376612 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:16.376827 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:16.376911 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 02:14:16.376995 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260809 02:14:16.379370 28779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.372444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:39909 (local address 127.0.0.1:53990)
0809 02:14:16.372544 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.372569 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:16.372611 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:16.372748 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:16.372748 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:16.372897 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:16.372944 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.372946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.373611 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.373612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.374129 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.374131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.374649 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.374650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.374728 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.375161 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:16.375166 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:16.376444 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:16.378322 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.378323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.378323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.378497 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.378715 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.378716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.378716 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.378780 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.379046 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:16.379047 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:16.379090 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 02:14:16.379174 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:14:16.379273 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 02:14:16.379379 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.372531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53990 (local address 127.5.9.189:39909)
0809 02:14:16.372604 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.372606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.372616 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:14:16.372676 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.372677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.372687 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.372744 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.373034 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.373500 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.374228 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.374543 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.374647 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.376532 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:16.376534 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:16.376534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:16.376542 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:16.378242 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.378571 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.378572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.378573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.378630 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.378853 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.378854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.378854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.378971 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:16.379041 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:16.379201 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 02:14:16.379278 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 02:14:16.380029 28777 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:14:16 dist-test-slave-p3t5 krb5kdc[28530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
Aug 09 02:14:16 dist-test-slave-p3t5 krb5kdc[28530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241656, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.188@KRBTEST.COM for kudu/127.5.9.188@KRBTEST.COM
I20260809 02:14:16.386694 28620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.378109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44092 (local address 127.5.9.190:34305)
0809 02:14:16.378166 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.378168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.378213 (+ 45us) server_negotiation.cc:407] Connection header received
0809 02:14:16.378289 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.378290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.378300 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.378344 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.378878 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.379357 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.380185 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.380470 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.380529 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.382952 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:16.382953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:16.382954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:16.382962 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:16.384730 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.385769 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.385770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.385770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.385851 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.386089 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.386090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.386091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.386245 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:16.386467 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:16.386505 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 02:14:16.386590 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 02:14:16.386694 28779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.380300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:35629 (local address 127.0.0.1:46148)
0809 02:14:16.380360 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.380370 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:16.380416 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:16.380547 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:16.380548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:16.380684 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:16.380710 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.380712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.381319 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.381320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.381912 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.381915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.382292 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.382292 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.382342 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.382761 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:16.382765 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:16.383832 (+ 1067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:16.385440 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.385441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.385442 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.385699 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.385961 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.385962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.385962 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.386021 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.386323 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:16.386323 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:16.386372 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 02:14:16.386447 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 02:14:16.386554 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 02:14:16.386706 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.380332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46148 (local address 127.5.9.188:35629)
0809 02:14:16.380364 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.380366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.380401 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:14:16.380470 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.380470 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.380478 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.380546 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.380781 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.381239 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.381995 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.382206 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.382292 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.383933 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:16.383934 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:16.383934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:16.383940 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:16.385240 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.385769 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.385770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.385770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.385889 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.386089 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.386090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.386090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.386244 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:16.386314 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:16.386491 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 02:14:16.386610 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":14}
I20260809 02:14:16.388221 28777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70: Bootstrap starting.
I20260809 02:14:16.389108 28777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:16.389406 28777 log.cc:824] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:16.390239 28777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70: No bootstrap required, opened a new log
Aug 09 02:14:16 dist-test-slave-p3t5 krb5kdc[28530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.189@KRBTEST.COM
I20260809 02:14:16.392242 28777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } }
I20260809 02:14:16.392407 28777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56a1130920b749e0b3288fd44a728c70, State: Initialized, Role: FOLLOWER
I20260809 02:14:16.392495 28777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } }
I20260809 02:14:16.392900 28777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:16.392915 28783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } } }
I20260809 02:14:16.393069 28783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 02:14:16.395134 28795 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:14:16.395260 28795 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:14:16.395789 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.387426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53996 (local address 127.5.9.189:39909)
0809 02:14:16.387492 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.387494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.387503 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:14:16.387526 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.387527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.387535 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.387586 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.388062 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.388607 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.389304 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.389541 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.389642 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.392248 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:16.392250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:16.392250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:16.392257 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:16.394376 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.394883 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.394885 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.394885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.394961 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.395219 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.395220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.395221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.395374 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:16.395545 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:16.395584 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 02:14:16.395667 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:14:16.397715 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:16.395635 28702 master_runner.cc:186] Error getting master registration for 127.5.9.190:34305: OK, instance_id { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" instance_seqno: 1786241656003950 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:14:16.398074 28702 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:14:16.398949 28589 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4eedc8ffba634d48b8d05319e59103f0"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44092
I20260809 02:14:16.411893 28694 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } attempt: 3
Aug 09 02:14:16 dist-test-slave-p3t5 krb5kdc[28530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241656, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.189@KRBTEST.COM for kudu/127.5.9.188@KRBTEST.COM
I20260809 02:14:16.420313 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.412381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46164 (local address 127.5.9.188:35629)
0809 02:14:16.412444 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.412446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.412460 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:14:16.412491 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.412492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.412502 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.412578 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.412867 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.413662 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.414790 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.415102 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.415179 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.417252 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:16.417253 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:16.417254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:16.417262 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:16.418989 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.419407 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.419408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.419409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.419470 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.419684 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.419685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.419685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.419888 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:16.419956 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:16.420117 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 02:14:16.420207 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 02:14:16.420315 28696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.412303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:35629 (local address 127.0.0.1:46164)
0809 02:14:16.412362 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.412386 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:16.412429 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:16.412600 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:16.412601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:16.412765 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:16.412815 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.412816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.413767 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.413768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.414676 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.414679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.415188 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.415189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.415247 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.415780 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:16.415786 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:16.417151 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:16.419081 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.419082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.419083 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.419327 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.419540 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.419541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.419542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.419616 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.419960 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:16.419961 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:16.420014 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 02:14:16.420090 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:14:16.420204 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 02:14:16.422533 28694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626: Bootstrap starting.
I20260809 02:14:16.423218 28694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:16.423604 28694 log.cc:824] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:16.424418 28694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626: No bootstrap required, opened a new log
I20260809 02:14:16.425977 28694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } }
I20260809 02:14:16.426121 28694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a6c803e9101443ea16a726a6fc4f626, State: Initialized, Role: FOLLOWER
I20260809 02:14:16.426218 28694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } }
I20260809 02:14:16.426530 28798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } } }
I20260809 02:14:16.426610 28694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:16.426625 28798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 02:14:16.428478 28809 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:14:16.428536 28809 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:14:16.428687 28625 master_runner.cc:186] Error getting master registration for 127.5.9.190:34305: OK, instance_id { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" instance_seqno: 1786241656003950 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:14:16.431754 28625 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:14:16.449134 28616 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } attempt: 3
Aug 09 02:14:16 dist-test-slave-p3t5 krb5kdc[28530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241655, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for kudu/127.5.9.188@KRBTEST.COM
I20260809 02:14:16.457410 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.449863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46176 (local address 127.5.9.188:35629)
0809 02:14:16.449926 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.449928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.449936 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:14:16.449965 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.449966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.449977 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.450040 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.450450 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.451240 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.451926 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.452220 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.452281 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.454543 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:16.454544 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:16.454545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:16.454552 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:16.456146 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.456547 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.456548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.456549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.456627 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.456845 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.456846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.456846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.456999 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:16.457062 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:16.457233 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 02:14:16.457308 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 02:14:16.457418 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.449570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:35629 (local address 127.0.0.1:46176)
0809 02:14:16.449666 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:16.449678 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:16.449731 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:16.450046 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:16.450047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:16.450312 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:16.450360 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.450362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.451319 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.451320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.451829 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:16.451830 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.452278 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.452279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:16.452334 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.452850 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:16.452855 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:16.454445 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:16.456242 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.456243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.456243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.456479 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.456699 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:16.456700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:16.456700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:16.456766 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:16.457052 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:16.457053 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:16.457103 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 02:14:16.457192 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 02:14:16.457309 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 02:14:16.459605 28616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0: Bootstrap starting.
I20260809 02:14:16.460386 28616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:16.460700 28616 log.cc:824] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:16.461637 28616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0: No bootstrap required, opened a new log
I20260809 02:14:16.463073 28616 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } }
I20260809 02:14:16.463223 28616 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eedc8ffba634d48b8d05319e59103f0, State: Initialized, Role: FOLLOWER
I20260809 02:14:16.463312 28616 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } }
I20260809 02:14:16.463668 28616 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:16.463689 28589 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 02:14:16.463737 28589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:16.464653 28589 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } }
I20260809 02:14:16.464939 28589 leader_election.cc:288] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [CANDIDATE]: Term 1 election: Requested vote from peers 4a6c803e9101443ea16a726a6fc4f626 (127.5.9.189:39909), 56a1130920b749e0b3288fd44a728c70 (127.5.9.188:35629)
I20260809 02:14:16.465807 28749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4eedc8ffba634d48b8d05319e59103f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "56a1130920b749e0b3288fd44a728c70"
I20260809 02:14:16.465969 28749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:16.466717 28749 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eedc8ffba634d48b8d05319e59103f0 in term 1.
W20260809 02:14:16.465747 28825 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 02:14:16.467094 28668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4eedc8ffba634d48b8d05319e59103f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4a6c803e9101443ea16a726a6fc4f626"
W20260809 02:14:16.467231 28825 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:14:16.467257 28668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:16.467994 28668 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eedc8ffba634d48b8d05319e59103f0 in term 1.
I20260809 02:14:16.468251 28562 leader_election.cc:302] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eedc8ffba634d48b8d05319e59103f0, 56a1130920b749e0b3288fd44a728c70; no voters:
I20260809 02:14:16.468598 28827 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:16.468814 28827 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [term 1 LEADER]: Becoming Leader. State: Replica: 4eedc8ffba634d48b8d05319e59103f0, State: Running, Role: LEADER
I20260809 02:14:16.468974 28827 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } }
I20260809 02:14:16.469563 28827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4eedc8ffba634d48b8d05319e59103f0. Latest consensus state: current_term: 1 leader_uuid: "4eedc8ffba634d48b8d05319e59103f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } } }
I20260809 02:14:16.469749 28827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:16.469993 28834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:16.470202 28834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:16.463635 28812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } } }
I20260809 02:14:16.472087 28812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:14:16.472421 28749 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70 [term 1 FOLLOWER]: Refusing update from remote peer 4eedc8ffba634d48b8d05319e59103f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:16.472797 28827 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:16.473315 28668 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626 [term 1 FOLLOWER]: Refusing update from remote peer 4eedc8ffba634d48b8d05319e59103f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:16.474257 28783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4eedc8ffba634d48b8d05319e59103f0. Latest consensus state: current_term: 1 leader_uuid: "4eedc8ffba634d48b8d05319e59103f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } } }
I20260809 02:14:16.474359 28783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:14:16.474365 28812 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:16.475682 28798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4eedc8ffba634d48b8d05319e59103f0. Latest consensus state: current_term: 1 leader_uuid: "4eedc8ffba634d48b8d05319e59103f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } } }
I20260809 02:14:16.475796 28798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:14:16.483470 28798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4eedc8ffba634d48b8d05319e59103f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } } }
I20260809 02:14:16.483594 28798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:14:16.483832 28839 mvcc.cc:205] Tried to move back new op lower bound from 7316445824907800576 to 7316445824898207744. Current Snapshot: MvccSnapshot[applied={T|T < 7316445824907800576}]
I20260809 02:14:16.484400 28620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.473095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44108 (local address 127.5.9.190:34305)
0809 02:14:16.473140 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.473143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.476102 (+ 2959us) server_negotiation.cc:407] Connection header received
0809 02:14:16.476216 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.476217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.476228 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.476260 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.477093 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.477855 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.478942 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.479235 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.479276 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.481002 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:16.481004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:16.481004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:16.481011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:16.482802 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.483231 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.483232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.483233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.483299 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.483544 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.483545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.483545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.483687 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:16.483767 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:16.484181 (+ 414us) server_negotiation.cc:299] Negotiation successful
0809 02:14:16.484278 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:14:16.489570 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.474662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54000 (local address 127.5.9.189:39909)
0809 02:14:16.475650 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.475652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.481778 (+ 6126us) server_negotiation.cc:407] Connection header received
0809 02:14:16.481883 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.481884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.481897 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.481928 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.482580 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.483306 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.484399 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.484704 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.484743 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.486450 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:16.486451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:16.486451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:16.486458 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:16.488171 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.488572 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.488574 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.488574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.488638 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.488860 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.488861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.488862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.489013 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:16.489106 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:16.489334 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 02:14:16.489443 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":967}
I20260809 02:14:16.490757 28835 mvcc.cc:205] Tried to move back new op lower bound from 7316445824907800576 to 7316445824898207744. Current Snapshot: MvccSnapshot[applied={T|T < 7316445824907800576}]
I20260809 02:14:16.490881 28827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4eedc8ffba634d48b8d05319e59103f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } } }
I20260809 02:14:16.490970 28827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:16.491015 28827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4eedc8ffba634d48b8d05319e59103f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } } }
I20260809 02:14:16.491053 28827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:16.491616 28783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4eedc8ffba634d48b8d05319e59103f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eedc8ffba634d48b8d05319e59103f0" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 34305 } } peers { permanent_uuid: "4a6c803e9101443ea16a726a6fc4f626" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 39909 } } peers { permanent_uuid: "56a1130920b749e0b3288fd44a728c70" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 35629 } } }
I20260809 02:14:16.491719 28783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:14:16.491816 28837 mvcc.cc:205] Tried to move back new op lower bound from 7316445824907800576 to 7316445824898207744. Current Snapshot: MvccSnapshot[applied={T|T < 7316445824907800576}]
I20260809 02:14:16.498993 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:16.476172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46184 (local address 127.5.9.188:35629)
0809 02:14:16.476210 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.476212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.484121 (+ 7909us) server_negotiation.cc:407] Connection header received
0809 02:14:16.484227 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.484229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.484241 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.484274 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.485728 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.486489 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.488092 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:16.488385 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:16.488430 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:16.492070 (+ 3640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:16.492071 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:16.492071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:16.492081 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:16.493980 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.498103 (+ 4123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.498104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.498104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.498166 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:16.498379 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:16.498380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:16.498380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:16.498519 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:16.498602 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:16.498806 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 02:14:16.498893 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:14:16.500147 28834 catalog_manager.cc:1347] Generated new cluster ID: 88eebf209a9449cf8266c8587b4bc5e2
I20260809 02:14:16.500201 28834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:16.521981 28834 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:16.522815 28834 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:16.531137 28834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4eedc8ffba634d48b8d05319e59103f0: Generated new TSK 0
I20260809 02:14:16.531347 28834 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:14:16.557843 28699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:16.556006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54006 (local address 127.5.9.189:39909)
0809 02:14:16.556056 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.556058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.557077 (+ 1019us) server_negotiation.cc:407] Connection header received
0809 02:14:16.557188 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.557189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.557205 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.557236 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.557760 (+ 524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54006: BlockingRecv error: recv got EOF from 127.0.0.1:54006 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 02:14:16.558663 28781 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:16.556380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46192 (local address 127.5.9.188:35629)
0809 02:14:16.556421 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.556423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.558087 (+ 1664us) server_negotiation.cc:407] Connection header received
0809 02:14:16.558188 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.558189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.558200 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.558230 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.558596 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46192: BlockingRecv error: recv got EOF from 127.0.0.1:46192 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 02:14:16.559446 28620 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:16.556784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44116 (local address 127.5.9.190:34305)
0809 02:14:16.556825 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.556827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.558924 (+ 2097us) server_negotiation.cc:407] Connection header received
0809 02:14:16.559022 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.559024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.559035 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.559063 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.559378 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44116: BlockingRecv error: recv got EOF from 127.0.0.1:44116 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 02:14:16.570900 28699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:16.570005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54012 (local address 127.5.9.189:39909)
0809 02:14:16.570060 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.570062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.570183 (+ 121us) server_negotiation.cc:407] Connection header received
0809 02:14:16.570303 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.570304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.570327 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.570359 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.570818 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54012: BlockingRecv error: recv got EOF from 127.0.0.1:54012 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 02:14:16.578598 28699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:16.573337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54018 (local address 127.5.9.189:39909)
0809 02:14:16.573374 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.573376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.577915 (+ 4539us) server_negotiation.cc:407] Connection header received
0809 02:14:16.578032 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.578034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.578047 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.578077 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.578523 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54018: BlockingRecv error: recv got EOF from 127.0.0.1:54018 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 02:14:16.583117 28699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:16.582114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54034 (local address 127.5.9.189:39909)
0809 02:14:16.582159 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.582161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.582508 (+ 347us) server_negotiation.cc:407] Connection header received
0809 02:14:16.582622 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.582624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.582636 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.582665 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.583056 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54034: BlockingRecv error: recv got EOF from 127.0.0.1:54034 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 02:14:16.589308 28699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:16.588381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54042 (local address 127.5.9.189:39909)
0809 02:14:16.588425 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.588427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.588730 (+ 303us) server_negotiation.cc:407] Connection header received
0809 02:14:16.588828 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.588829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.588852 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.588881 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.589248 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54042: BlockingRecv error: recv got EOF from 127.0.0.1:54042 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 02:14:16.599485 28699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:16.598675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54044 (local address 127.5.9.189:39909)
0809 02:14:16.598727 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.598730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.598872 (+ 142us) server_negotiation.cc:407] Connection header received
0809 02:14:16.598976 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.598978 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.598990 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.599041 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.599393 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54044: BlockingRecv error: recv got EOF from 127.0.0.1:54044 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 02:14:16.617470 28699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:16.616645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54054 (local address 127.5.9.189:39909)
0809 02:14:16.616697 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.616699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.616837 (+ 138us) server_negotiation.cc:407] Connection header received
0809 02:14:16.616916 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.616917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.616931 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.617007 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.617371 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54054: BlockingRecv error: recv got EOF from 127.0.0.1:54054 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 02:14:16.651623 28699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:16.650729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54066 (local address 127.5.9.189:39909)
0809 02:14:16.650808 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.650810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.650972 (+ 162us) server_negotiation.cc:407] Connection header received
0809 02:14:16.651048 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.651049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.651061 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.651115 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.651550 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54066: BlockingRecv error: recv got EOF from 127.0.0.1:54066 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 02:14:16.717738 28699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:16.716880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54072 (local address 127.5.9.189:39909)
0809 02:14:16.716973 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.716976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.717078 (+ 102us) server_negotiation.cc:407] Connection header received
0809 02:14:16.717173 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.717174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.717186 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.717257 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.717649 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54072: BlockingRecv error: recv got EOF from 127.0.0.1:54072 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 02:14:16.848008 28699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:16.847177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54074 (local address 127.5.9.189:39909)
0809 02:14:16.847269 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:16.847272 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:16.847382 (+ 110us) server_negotiation.cc:407] Connection header received
0809 02:14:16.847489 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:16.847490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:16.847503 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:16.847581 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:16.847934 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54074: BlockingRecv error: recv got EOF from 127.0.0.1:54074 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 02:14:17.106088 28699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:17.105303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54088 (local address 127.5.9.189:39909)
0809 02:14:17.105398 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:17.105400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:17.105412 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:14:17.105511 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:17.105512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:17.105525 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:17.105604 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:17.106014 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54088: BlockingRecv error: recv got EOF from 127.0.0.1:54088 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 02:14:17.395792 28795 catalog_manager.cc:1259] Loaded cluster ID: 88eebf209a9449cf8266c8587b4bc5e2
I20260809 02:14:17.395879 28795 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70: loading cluster ID for follower catalog manager: success
I20260809 02:14:17.396636 28795 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70: acquiring CA information for follower catalog manager: success
I20260809 02:14:17.396924 28795 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 56a1130920b749e0b3288fd44a728c70: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:14:17.428983 28809 catalog_manager.cc:1259] Loaded cluster ID: 88eebf209a9449cf8266c8587b4bc5e2
I20260809 02:14:17.429065 28809 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626: loading cluster ID for follower catalog manager: success
I20260809 02:14:17.429940 28809 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626: acquiring CA information for follower catalog manager: success
I20260809 02:14:17.430227 28809 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4a6c803e9101443ea16a726a6fc4f626: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:14:17.622265 28915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:17.619465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54096 (local address 127.5.9.189:39909)
0809 02:14:17.619598 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:17.619601 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:17.619649 (+ 48us) server_negotiation.cc:407] Connection header received
0809 02:14:17.619745 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:17.619746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:17.619821 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:17.619919 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:17.620085 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:17.620609 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:17.621395 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:17.621633 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:17.621715 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:17.621892 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:17.622014 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:17.622107 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:17.622131 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 02:14:17.622155 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":58,"threads_started":1}
I20260809 02:14:17.626777 28921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:17.624103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46198 (local address 127.5.9.188:35629)
0809 02:14:17.624209 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:17.624211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:17.624220 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:14:17.624286 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:17.624287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:17.624337 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:17.624435 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:17.624556 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:17.625041 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:17.625843 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:17.626137 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:17.626226 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:17.626379 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:17.626499 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:17.626584 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:17.626633 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 02:14:17.626656 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":53,"threads_started":1}
I20260809 02:14:17.631387 28915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:17.628539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54100 (local address 127.5.9.189:39909)
0809 02:14:17.628591 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:17.628594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:17.628713 (+ 119us) server_negotiation.cc:407] Connection header received
0809 02:14:17.628861 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:17.628863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:17.628883 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:17.628931 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:17.629044 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:17.629801 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:17.630660 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:17.630916 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:17.630978 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:17.631105 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:17.631140 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:17.631192 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:17.631255 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 02:14:17.631280 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:14:17.631682 28921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:17.628793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46200 (local address 127.5.9.188:35629)
0809 02:14:17.628986 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:17.628988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:17.628997 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:14:17.629017 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:17.629018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:17.629029 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:17.629076 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:17.629256 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:17.629802 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:17.630889 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:17.631156 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:17.631212 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:17.631390 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:17.631424 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:17.631452 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:17.631563 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 02:14:17.631589 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260809 02:14:17.632077 28928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:17.628543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44132 (local address 127.5.9.190:34305)
0809 02:14:17.628664 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:17.628667 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:17.628791 (+ 124us) server_negotiation.cc:407] Connection header received
0809 02:14:17.628907 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:17.628908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:17.628966 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:17.629050 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:17.629247 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:17.630024 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:17.631053 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:17.631382 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:17.631444 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:17.631731 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:17.631845 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:17.631912 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:17.631967 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 02:14:17.631990 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
I20260809 02:14:17.633596 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 28545
I20260809 02:14:17.641523 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 28625
I20260809 02:14:17.648324 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 28702
2026-08-09T02:14:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2977 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:17 dist-test-slave-p3t5 krb5kdc[28932](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:17 dist-test-slave-p3t5 krb5kdc[28932](info): set up 2 sockets
Aug 09 02:14:17 dist-test-slave-p3t5 krb5kdc[28932](info): commencing operation
krb5kdc: starting...
W20260809 02:14:18.754316 5158 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/test-user.keytab.
Aug 09 02:14:18 dist-test-slave-p3t5 krb5kdc[28932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.9.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
Entry for principal kudu/127.5.9.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab.
I20260809 02:14:18.887156 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:37107
--webserver_interface=127.5.9.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:37107,127.5.9.189:33545,127.5.9.188:33345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:18.972642 28945 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:14:18.972872 28945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:18.972891 28945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:18.972913 28945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:18.974071 28945 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:18.974107 28945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:18.974121 28945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:18.974134 28945 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:18.974148 28945 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:18.975596 28945 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:37107,127.5.9.189:33545,127.5.9.188:33345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.190.keytab
--principal=kudu/127.5.9.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.190:37107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.9.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:18.975806 28945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:18.975999 28945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:18.981105 28950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:18.981078 28951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:18.981187 28945 server_base.cc:1034] running on GCE node
W20260809 02:14:18.981096 28953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:18.981544 28945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:18.981931 28945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:18.983084 28945 hybrid_clock.cc:648] HybridClock initialized: now 1786241658983063 us; error 42 us; skew 500 ppm
Aug 09 02:14:18 dist-test-slave-p3t5 krb5kdc[28932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241658, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:18.990326 28945 init.cc:377] Logged in from keytab as kudu/127.5.9.190@KRBTEST.COM (short username kudu)
I20260809 02:14:18.990881 28945 webserver.cc:460] Webserver started at http://127.5.9.190:37063/ using document root <none> and password file <none>
I20260809 02:14:18.991139 28945 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:18.991211 28945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:18.991379 28945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:18.992556 28945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/data/instance:
uuid: "121bc67902f44f3085e4cae8fecb18f8"
format_stamp: "Formatted at 2026-08-09 02:14:18 on dist-test-slave-p3t5"
server_key: "220bdbe54242e5f9e9829ad2f94015f2"
server_key_iv: "2d04f6546e7b343750f4976f80f65880"
server_key_version: "encryptionkey@0"
I20260809 02:14:18.992965 28945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/wal/instance:
uuid: "121bc67902f44f3085e4cae8fecb18f8"
format_stamp: "Formatted at 2026-08-09 02:14:18 on dist-test-slave-p3t5"
server_key: "220bdbe54242e5f9e9829ad2f94015f2"
server_key_iv: "2d04f6546e7b343750f4976f80f65880"
server_key_version: "encryptionkey@0"
I20260809 02:14:18.994566 28945 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:18.995446 28960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:18.995649 28945 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:18.995764 28945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/wal
uuid: "121bc67902f44f3085e4cae8fecb18f8"
format_stamp: "Formatted at 2026-08-09 02:14:18 on dist-test-slave-p3t5"
server_key: "220bdbe54242e5f9e9829ad2f94015f2"
server_key_iv: "2d04f6546e7b343750f4976f80f65880"
server_key_version: "encryptionkey@0"
I20260809 02:14:18.995880 28945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:19.010428 28945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:19.012706 28945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:19.012916 28945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:19.017395 28945 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.190:37107
I20260809 02:14:19.017434 29014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.190:37107 every 8 connection(s)
I20260809 02:14:19.017894 28945 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:19.018997 29015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:19.020709 29015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:14:19.023372 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 28945
I20260809 02:14:19.023505 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-0/wal/instance
I20260809 02:14:19.023720 5158 external_mini_cluster.cc:1428] Setting key 0821f1cf6868cfd3c3a8b0f8d36a3fd8
Aug 09 02:14:19 dist-test-slave-p3t5 krb5kdc[28932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241658, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.9.189@KRBTEST.COM; defaulting to no policy
I20260809 02:14:19.032594 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.021508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54296 (local address 127.5.9.190:37107)
0809 02:14:19.021757 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.021762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.021871 (+ 109us) server_negotiation.cc:407] Connection header received
0809 02:14:19.022389 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.022391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.022430 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.022513 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.023550 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.024878 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.025910 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.026443 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.026566 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.029162 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:19.029169 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:19.029169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:19.029183 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:19.030869 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.031297 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.031300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.031301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.031398 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.031638 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.031639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.031639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.031876 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:19.031969 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:19.032173 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 02:14:19.032329 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":152,"threads_started":1}
I20260809 02:14:19.032609 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.021449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:37107 (local address 127.0.0.1:54296)
0809 02:14:19.021658 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.021718 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:19.022248 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:19.022576 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:19.022579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:19.022904 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:19.023374 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.023381 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.025009 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.025011 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.025814 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.025817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.026582 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.026583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.026648 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.027276 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:19.027287 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:19.029059 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:19.030977 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.030984 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.030986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.031195 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.031471 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.031472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.031472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.031569 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.031960 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:19.031966 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:19.032035 (+ 69us) client_negotiation.cc:769] Sending connection context
0809 02:14:19.032146 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 02:14:19.032399 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":96,"thread_start_us":69,"threads_started":1}
I20260809 02:14:19.033345 29015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:14:19.033802 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.033606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:33545 (local address 127.0.0.1:58956)
0809 02:14:19.033688 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.033762 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:33545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 02:14:19.033975 28961 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.9.189:33545: connect: Connection refused (error 111)
I20260809 02:14:19.034407 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.034314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:33545 (local address 127.0.0.1:58966)
0809 02:14:19.034362 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.034389 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:33545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 02:14:19.034556 29015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.189:33545: Network error: Client connection negotiation failed: client connection to 127.5.9.189:33545: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.189.keytab.
Entry for principal kudu/127.5.9.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.189.keytab.
I20260809 02:14:19.052951 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.189:33545
--webserver_interface=127.5.9.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.189.keytab
--principal=kudu/127.5.9.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:37107,127.5.9.189:33545,127.5.9.188:33345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:14:19.083782 29015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } attempt: 1
I20260809 02:14:19.084347 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.084198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:33545 (local address 127.0.0.1:58980)
0809 02:14:19.084272 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.084312 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:33545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 02:14:19.084820 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.084729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:33545 (local address 127.0.0.1:58994)
0809 02:14:19.084772 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.084800 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:33545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 02:14:19.084936 29015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.189:33545: Network error: Client connection negotiation failed: client connection to 127.5.9.189:33545: connect: Connection refused (error 111)
W20260809 02:14:19.139297 29023 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:14:19.139572 29023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:19.139606 29023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:19.139652 29023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:19.140844 29023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:19.140923 29023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:19.140945 29023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:19.140964 29023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:19.140993 29023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:19.142616 29023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:37107,127.5.9.189:33545,127.5.9.188:33345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.189.keytab
--principal=kudu/127.5.9.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.189:33545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.9.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:19.142928 29023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:19.143183 29023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:19.148343 29029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:19.148407 29023 server_base.cc:1034] running on GCE node
W20260809 02:14:19.148340 29028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:19.148340 29031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:19.148802 29023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:19.149128 29023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:19.150293 29023 hybrid_clock.cc:648] HybridClock initialized: now 1786241659150270 us; error 31 us; skew 500 ppm
Aug 09 02:14:19 dist-test-slave-p3t5 krb5kdc[28932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241659, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:19.153092 29023 init.cc:377] Logged in from keytab as kudu/127.5.9.189@KRBTEST.COM (short username kudu)
I20260809 02:14:19.153075 29015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } attempt: 2
I20260809 02:14:19.153517 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.153377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:33545 (local address 127.0.0.1:58996)
0809 02:14:19.153450 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.153487 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:33545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 02:14:19.153764 29023 webserver.cc:460] Webserver started at http://127.5.9.189:39647/ using document root <none> and password file <none>
I20260809 02:14:19.153913 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.153820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:33545 (local address 127.0.0.1:59012)
0809 02:14:19.153854 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.153879 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.189:33545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260809 02:14:19.154002 29015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.189:33545: Network error: Client connection negotiation failed: client connection to 127.5.9.189:33545: connect: Connection refused (error 111)
I20260809 02:14:19.154101 29023 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:19.154167 29023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:19.154318 29023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:19.155450 29023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/data/instance:
uuid: "1ba25c4eb02149668cf1376bc524668e"
format_stamp: "Formatted at 2026-08-09 02:14:19 on dist-test-slave-p3t5"
server_key: "d418ab042409825b9909d2039bc8cf56"
server_key_iv: "8bb1c87152d29a04569281193fd9e772"
server_key_version: "encryptionkey@0"
I20260809 02:14:19.155838 29023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/wal/instance:
uuid: "1ba25c4eb02149668cf1376bc524668e"
format_stamp: "Formatted at 2026-08-09 02:14:19 on dist-test-slave-p3t5"
server_key: "d418ab042409825b9909d2039bc8cf56"
server_key_iv: "8bb1c87152d29a04569281193fd9e772"
server_key_version: "encryptionkey@0"
I20260809 02:14:19.157266 29023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:19.158161 29039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:19.158378 29023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:14:19.158496 29023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/wal
uuid: "1ba25c4eb02149668cf1376bc524668e"
format_stamp: "Formatted at 2026-08-09 02:14:19 on dist-test-slave-p3t5"
server_key: "d418ab042409825b9909d2039bc8cf56"
server_key_iv: "8bb1c87152d29a04569281193fd9e772"
server_key_version: "encryptionkey@0"
I20260809 02:14:19.158579 29023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:19.165258 29023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:19.167760 29023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:19.167941 29023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:19.172235 29023 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.189:33545
I20260809 02:14:19.172269 29092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.189:33545 every 8 connection(s)
I20260809 02:14:19.172643 29023 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/data/info.pb
I20260809 02:14:19.173698 29093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:19.175240 29093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:14:19.179044 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 29023
I20260809 02:14:19.179180 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-1/wal/instance
I20260809 02:14:19.179405 5158 external_mini_cluster.cc:1428] Setting key fe32812e0e23a871b323f829b1e2e57c
Aug 09 02:14:19 dist-test-slave-p3t5 krb5kdc[28932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241659, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.189@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:14:19.185424 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.176001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54306 (local address 127.5.9.190:37107)
0809 02:14:19.176100 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.176102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.176264 (+ 162us) server_negotiation.cc:407] Connection header received
0809 02:14:19.176794 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.176796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.176812 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.176865 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.177650 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.178159 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.179238 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.179572 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.179661 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.181981 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:19.181983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:19.181983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:19.181992 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:19.183568 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.184212 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.184214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.184214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.184289 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.184620 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.184647 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.184648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.184829 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:19.184891 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:19.185272 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 02:14:19.185352 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:14:19.185813 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.175923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:37107 (local address 127.0.0.1:54306)
0809 02:14:19.176142 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.176179 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:19.176674 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:19.176953 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:19.176957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:19.177292 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:19.177524 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.177529 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.178284 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.178285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.179124 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.179126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.179677 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.179679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.179791 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.180374 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:19.180383 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:19.181881 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:19.183677 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.183706 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.183709 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.184096 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.184371 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.184372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.184372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.184521 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.184912 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:19.184921 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:19.185110 (+ 189us) client_negotiation.cc:769] Sending connection context
0809 02:14:19.185250 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 02:14:19.185455 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260809 02:14:19.186547 29093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.9.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.9.188@KRBTEST.COM" created.
Aug 09 02:14:19 dist-test-slave-p3t5 krb5kdc[28932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241659, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.189@KRBTEST.COM for kudu/127.5.9.189@KRBTEST.COM
I20260809 02:14:19.196283 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.186860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:33545 (local address 127.0.0.1:59020)
0809 02:14:19.187114 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.187127 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:19.187184 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:19.187348 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:19.187349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:19.187521 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:19.187592 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.187595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.188459 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.188460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.189312 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.189315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.189928 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.189929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.189999 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.190512 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:19.190517 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:19.192756 (+ 2239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:19.195104 (+ 2348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.195105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.195106 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.195277 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.195515 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.195516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.195516 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.195576 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.195892 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:19.195892 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:19.195932 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 02:14:19.196023 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 02:14:19.196162 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237}
I20260809 02:14:19.196352 29097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.186939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59020 (local address 127.5.9.189:33545)
0809 02:14:19.187127 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.187131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.187179 (+ 48us) server_negotiation.cc:407] Connection header received
0809 02:14:19.187243 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.187245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.187256 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.187319 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.187667 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.188367 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.189411 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.189853 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.189921 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.192871 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:19.192881 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:19.192881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:19.192887 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:19.195003 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.195370 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.195374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.195375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.195442 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.195652 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.195652 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.195653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.195817 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:19.195923 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:19.196058 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 02:14:19.196225 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":160,"thread_start_us":114,"threads_started":1}
I20260809 02:14:19.197062 29093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:14:19.197468 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.197323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:33345 (local address 127.0.0.1:55558)
0809 02:14:19.197380 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.197433 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 02:14:19.197680 29040 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
I20260809 02:14:19.198199 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.198089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:33345 (local address 127.0.0.1:55566)
0809 02:14:19.198140 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.198170 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 02:14:19.198307 29093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:33345: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.9.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.188.keytab.
Entry for principal kudu/127.5.9.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.188.keytab.
I20260809 02:14:19.209442 5158 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktks196/build/release/bin/kudu
/tmp/dist-test-tasktks196/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.188:33345
--webserver_interface=127.5.9.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.188.keytab
--principal=kudu/127.5.9.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.9.148:36743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.9.190:37107,127.5.9.189:33545,127.5.9.188:33345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:14:19.245195 29015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } attempt: 3
I20260809 02:14:19.247565 29093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } attempt: 1
I20260809 02:14:19.248025 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.247868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:33345 (local address 127.0.0.1:55574)
0809 02:14:19.247930 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.247976 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 02:14:19.248404 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.248287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:33345 (local address 127.0.0.1:55588)
0809 02:14:19.248348 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.248383 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 02:14:19.248508 29093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:33345: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
Aug 09 02:14:19 dist-test-slave-p3t5 krb5kdc[28932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241658, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for kudu/127.5.9.189@KRBTEST.COM
I20260809 02:14:19.253156 29097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.245762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59022 (local address 127.5.9.189:33545)
0809 02:14:19.245870 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.245873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.245885 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:14:19.245917 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.245918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.245935 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.246007 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.246377 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.246910 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.247677 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.247993 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.248071 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.250400 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:19.250402 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:19.250402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:19.250410 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:19.251945 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.252319 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.252321 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.252322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.252385 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.252601 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.252602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.252602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.252730 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:19.252811 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:19.252980 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 02:14:19.253064 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:14:19.253161 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.245681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:33545 (local address 127.0.0.1:59022)
0809 02:14:19.245739 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.245761 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:19.245835 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:19.246018 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:19.246019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:19.246207 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:19.246279 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.246281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.247006 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.247008 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.247582 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.247584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.248089 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.248090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.248200 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.248797 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:19.248803 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:19.250298 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:19.252036 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.252037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.252038 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.252227 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.252463 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.252464 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.252464 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.252532 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.252803 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:19.252804 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:19.252858 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 02:14:19.252942 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:14:19.253059 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 02:14:19.253808 29015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:14:19.254168 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.254076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:33345 (local address 127.0.0.1:55592)
0809 02:14:19.254115 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.254148 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 02:14:19.254544 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.254429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:33345 (local address 127.0.0.1:55598)
0809 02:14:19.254477 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.254512 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 02:14:19.254657 29015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:33345: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
I20260809 02:14:19.285840 29015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } attempt: 1
I20260809 02:14:19.286375 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.286217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:33345 (local address 127.0.0.1:55600)
0809 02:14:19.286285 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.286338 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 02:14:19.286813 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.286676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:33345 (local address 127.0.0.1:55604)
0809 02:14:19.286750 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.286786 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 02:14:19.286921 29015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:33345: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
W20260809 02:14:19.300194 29100 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:14:19.300431 29100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:19.300449 29100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:19.300472 29100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:19.301601 29100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:19.301713 29100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:19.301739 29100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:19.301754 29100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:19.301775 29100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:19.303289 29100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.9.148:36743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.9.190:37107,127.5.9.189:33545,127.5.9.188:33345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/krb5kdc/kudu_127.5.9.188.keytab
--principal=kudu/127.5.9.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.9.188:33345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.9.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:19.303500 29100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:19.303686 29100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:19.308663 29108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:19.308799 29106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:19.308663 29105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:19.308921 29100 server_base.cc:1034] running on GCE node
I20260809 02:14:19.309214 29100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:19.309497 29100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:19.310643 29100 hybrid_clock.cc:648] HybridClock initialized: now 1786241659310645 us; error 47 us; skew 500 ppm
Aug 09 02:14:19 dist-test-slave-p3t5 krb5kdc[28932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241659, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:19.313344 29100 init.cc:377] Logged in from keytab as kudu/127.5.9.188@KRBTEST.COM (short username kudu)
I20260809 02:14:19.313865 29100 webserver.cc:460] Webserver started at http://127.5.9.188:36195/ using document root <none> and password file <none>
I20260809 02:14:19.314091 29100 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:19.314136 29100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:19.314231 29100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:19.315402 29100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/data/instance:
uuid: "66cbb472d379445eb8db3b94be7e5da3"
format_stamp: "Formatted at 2026-08-09 02:14:19 on dist-test-slave-p3t5"
server_key: "250ab5ffbe349d145dfbc9bc845a04ed"
server_key_iv: "eb04e011849adb259256509dce6cbe6a"
server_key_version: "encryptionkey@0"
I20260809 02:14:19.315727 29100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/wal/instance:
uuid: "66cbb472d379445eb8db3b94be7e5da3"
format_stamp: "Formatted at 2026-08-09 02:14:19 on dist-test-slave-p3t5"
server_key: "250ab5ffbe349d145dfbc9bc845a04ed"
server_key_iv: "eb04e011849adb259256509dce6cbe6a"
server_key_version: "encryptionkey@0"
I20260809 02:14:19.316648 29093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } attempt: 2
I20260809 02:14:19.317081 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.316939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:33345 (local address 127.0.0.1:55606)
0809 02:14:19.317011 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.317047 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 02:14:19.317233 29100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:19.317476 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.317372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:33345 (local address 127.0.0.1:55608)
0809 02:14:19.317419 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.317449 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 02:14:19.317579 29093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:33345: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
I20260809 02:14:19.318208 29115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:19.318392 29100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:14:19.318488 29100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/data,/tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/wal
uuid: "66cbb472d379445eb8db3b94be7e5da3"
format_stamp: "Formatted at 2026-08-09 02:14:19 on dist-test-slave-p3t5"
server_key: "250ab5ffbe349d145dfbc9bc845a04ed"
server_key_iv: "eb04e011849adb259256509dce6cbe6a"
server_key_version: "encryptionkey@0"
I20260809 02:14:19.318578 29100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:19.358983 29100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:19.361235 29100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:19.361425 29100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:19.362061 29015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } attempt: 2
I20260809 02:14:19.362558 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.362377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:33345 (local address 127.0.0.1:55618)
0809 02:14:19.362456 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.362509 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 02:14:19.363112 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.362972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:33345 (local address 127.0.0.1:55626)
0809 02:14:19.363051 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.363085 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 02:14:19.363250 29015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.9.188:33345: Network error: Client connection negotiation failed: client connection to 127.5.9.188:33345: connect: Connection refused (error 111)
I20260809 02:14:19.366024 29100 rpc_server.cc:307] RPC server started. Bound to: 127.5.9.188:33345
I20260809 02:14:19.366110 29168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.9.188:33345 every 8 connection(s)
I20260809 02:14:19.366493 29100 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/data/info.pb
I20260809 02:14:19.367612 29169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:19.369294 29169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:14:19 dist-test-slave-p3t5 krb5kdc[28932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241659, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.188@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
I20260809 02:14:19.376353 5158 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktks196/build/release/bin/kudu as pid 29100
I20260809 02:14:19.376519 5158 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktks196/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241377393328-5158-0/minicluster-data/master-2/wal/instance
I20260809 02:14:19.376729 5158 external_mini_cluster.cc:1428] Setting key 0f209fd5941eb73e77d1e396ae702ec7
I20260809 02:14:19.379310 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.370109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54316 (local address 127.5.9.190:37107)
0809 02:14:19.370157 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.370160 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.370410 (+ 250us) server_negotiation.cc:407] Connection header received
0809 02:14:19.370870 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.370871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.370885 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.370961 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.371616 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.372139 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.373138 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.373340 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.373398 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.375824 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:19.375827 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:19.375827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:19.375834 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:19.377846 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.378252 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.378253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.378254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.378332 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.378600 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.378602 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.378602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.378750 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:19.378808 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:19.379163 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 02:14:19.379235 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:14:19.379799 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.370065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.190:37107 (local address 127.0.0.1:54316)
0809 02:14:19.370247 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.370294 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:19.370768 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:19.371018 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:19.371022 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:19.371298 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:19.371502 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.371506 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.372230 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.372230 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.373055 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.373057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.373433 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.373434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.373545 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.374200 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:19.374209 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:19.375703 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:19.377938 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.377943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.377945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.378172 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.378410 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.378410 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.378411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.378520 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.378821 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:19.378826 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:19.379012 (+ 186us) client_negotiation.cc:769] Sending connection context
0809 02:14:19.379130 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 02:14:19.379530 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":215,"thread_start_us":103,"threads_started":1}
I20260809 02:14:19.380426 29169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:14:19 dist-test-slave-p3t5 krb5kdc[28932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.188@KRBTEST.COM
Aug 09 02:14:19 dist-test-slave-p3t5 krb5kdc[28932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241659, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.188@KRBTEST.COM for kudu/127.5.9.189@KRBTEST.COM
I20260809 02:14:19.386931 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.377213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55636 (local address 127.5.9.188:33345)
0809 02:14:19.377599 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.377603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.377665 (+ 62us) server_negotiation.cc:407] Connection header received
0809 02:14:19.377692 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.377694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.377707 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.377766 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.378601 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.379604 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.380593 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.381078 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.381153 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.383871 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:19.383883 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:19.383883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:19.383890 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:19.385564 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.385995 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.385998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.385999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.386064 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.386290 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.386291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.386291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.386455 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:19.386624 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:19.386710 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 02:14:19.386811 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":114,"threads_started":1}
I20260809 02:14:19.388427 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.380698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.189:33545 (local address 127.0.0.1:59028)
0809 02:14:19.380736 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.380745 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:19.380775 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:19.380991 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:19.380992 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:19.381155 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:19.381214 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.381216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.382254 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.382255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.382766 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.382768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.383164 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.383166 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.383239 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.383723 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:19.383728 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:19.385326 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:19.387232 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.387233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.387233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.387487 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.387698 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.387699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.387699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.387784 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.388065 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:19.388066 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:19.388114 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 02:14:19.388213 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 02:14:19.388325 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 02:14:19.388447 29097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.380797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59028 (local address 127.5.9.189:33545)
0809 02:14:19.380866 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.380868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.380879 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:14:19.380909 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.380910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.380919 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.380968 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.381300 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.382163 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.382865 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.383088 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.383167 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.385437 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:19.385439 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:19.385440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:19.385448 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:19.387148 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.387553 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.387555 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.387556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.387623 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.387841 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.387842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.387843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.388000 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:19.388070 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:19.388264 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 02:14:19.388348 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:14:19.388908 29169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:14:19 dist-test-slave-p3t5 krb5kdc[28932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.190@KRBTEST.COM
Aug 09 02:14:19 dist-test-slave-p3t5 krb5kdc[28932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241659, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.188@KRBTEST.COM for kudu/127.5.9.188@KRBTEST.COM
I20260809 02:14:19.396286 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.387816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54328 (local address 127.5.9.190:37107)
0809 02:14:19.387873 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.387876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.387884 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:14:19.387909 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.387910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.387923 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.387984 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.388441 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.389349 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.390361 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.390596 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.390647 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.393327 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:19.393329 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:19.393329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:19.393338 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:19.394955 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.395355 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.395356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.395357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.395423 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.395650 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.395651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.395651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.395791 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:19.396007 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:19.396049 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 02:14:19.396131 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 02:14:19.396401 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.389271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55638 (local address 127.5.9.188:33345)
0809 02:14:19.389309 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.389312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.389321 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:14:19.389386 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.389387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.389399 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.389457 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.389763 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.390290 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.391214 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.391461 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.391516 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.393710 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:19.393711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:19.393712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:19.393719 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:19.395176 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.395536 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.395537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.395537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.395596 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.395797 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.395798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.395799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.395914 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:19.395988 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:19.396138 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 02:14:19.396313 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":18}
I20260809 02:14:19.396696 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.389213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:33345 (local address 127.0.0.1:55638)
0809 02:14:19.389266 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.389277 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:19.389319 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:19.389459 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:19.389459 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:19.389646 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:19.389683 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.389685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.390392 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.390393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.391129 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.391132 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.391543 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.391545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.391624 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.391979 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:19.391985 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:19.393497 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:19.395283 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.395284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.395284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.395463 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.395673 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.395674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.395674 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.395736 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.395992 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:19.395993 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:19.396031 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 02:14:19.396115 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:14:19.396263 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":4}
I20260809 02:14:19.398604 29169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3: Bootstrap starting.
I20260809 02:14:19.399715 29169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:19.400085 29169 log.cc:824] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:19.401192 29169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3: No bootstrap required, opened a new log
Aug 09 02:14:19 dist-test-slave-p3t5 krb5kdc[28932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.9.189@KRBTEST.COM
I20260809 02:14:19.402902 29169 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } }
I20260809 02:14:19.403100 29169 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66cbb472d379445eb8db3b94be7e5da3, State: Initialized, Role: FOLLOWER
I20260809 02:14:19.403195 29169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } }
I20260809 02:14:19.403566 29175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } } }
I20260809 02:14:19.403631 29169 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:19.403713 29175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 02:14:19.405817 29189 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:14:19.405880 29189 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:14:19.406530 29097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.397432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59042 (local address 127.5.9.189:33545)
0809 02:14:19.397496 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.397498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.397750 (+ 252us) server_negotiation.cc:407] Connection header received
0809 02:14:19.397808 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.397809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.397822 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.397887 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.398442 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.399287 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.400069 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.400423 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.400505 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.403019 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:19.403021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:19.403021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:19.403031 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:19.404929 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.405371 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.405372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.405373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.405450 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.405711 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.405713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.405713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.405881 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:19.406072 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:19.406330 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 02:14:19.406427 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:14:19.408142 5158 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:19.408746 29093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } attempt: 3
I20260809 02:14:19.406203 29100 master_runner.cc:186] Error getting master registration for 127.5.9.190:37107: OK, instance_id { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" instance_seqno: 1786241659016102 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:14:19.409256 29100 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:14:19.408989 28988 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "121bc67902f44f3085e4cae8fecb18f8"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54328
Aug 09 02:14:19 dist-test-slave-p3t5 krb5kdc[28932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241659, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.189@KRBTEST.COM for kudu/127.5.9.188@KRBTEST.COM
I20260809 02:14:19.416612 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.408992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:33345 (local address 127.0.0.1:55650)
0809 02:14:19.409698 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.409712 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:19.409761 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:19.410280 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:19.410281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:19.410443 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:19.410484 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.410486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.411134 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.411135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.411657 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.411659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.412051 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.412052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.412110 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.412591 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:19.412596 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:19.413834 (+ 1238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:19.415515 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.415516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.415517 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.415697 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.415913 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.415913 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.415914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.415981 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.416277 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:19.416278 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:19.416325 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 02:14:19.416412 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 02:14:19.416516 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684}
I20260809 02:14:19.416612 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.409855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55650 (local address 127.5.9.188:33345)
0809 02:14:19.410123 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.410125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.410134 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:14:19.410175 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.410176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.410188 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.410240 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.410563 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.411046 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.411745 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.411960 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.412049 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.413933 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:19.413934 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:19.413935 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:19.413942 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:19.415415 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.415772 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.415773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.415773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.415837 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.416046 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.416047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.416047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.416197 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:19.416273 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:19.416437 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 02:14:19.416518 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245}
I20260809 02:14:19.418008 29093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e: Bootstrap starting.
I20260809 02:14:19.418910 29093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:19.419150 29093 log.cc:824] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:19.419914 29093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e: No bootstrap required, opened a new log
I20260809 02:14:19.421484 29093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } }
I20260809 02:14:19.421605 29093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ba25c4eb02149668cf1376bc524668e, State: Initialized, Role: FOLLOWER
I20260809 02:14:19.421739 29093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } }
I20260809 02:14:19.422014 29192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } } }
I20260809 02:14:19.422093 29093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:19.422223 29192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e [sys.catalog]: This master's current role is: FOLLOWER
W20260809 02:14:19.424116 29203 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:14:19.424175 29203 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:14:19.424434 29023 master_runner.cc:186] Error getting master registration for 127.5.9.190:37107: OK, instance_id { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" instance_seqno: 1786241659016102 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:14:19.427380 29023 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:14:19.462422 29015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } attempt: 3
Aug 09 02:14:19 dist-test-slave-p3t5 krb5kdc[28932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241658, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.9.190@KRBTEST.COM for kudu/127.5.9.188@KRBTEST.COM
I20260809 02:14:19.471258 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.462921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55666 (local address 127.5.9.188:33345)
0809 02:14:19.463027 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.463029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.463038 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:14:19.463067 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.463068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.463078 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.463131 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.463554 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.464325 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.465369 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.465698 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.465768 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.468230 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:19.468231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:19.468232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:19.468241 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:19.470019 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.470443 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.470444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.470444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.470513 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.470713 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.470714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.470714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.470859 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:19.470929 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:19.471070 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 02:14:19.471153 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 02:14:19.471271 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.462825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.9.188:33345 (local address 127.0.0.1:55666)
0809 02:14:19.462886 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:19.462899 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:19.462944 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:19.463163 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:19.463164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:19.463377 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:19.463438 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.463441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.464426 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.464427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.465260 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:19.465263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.465784 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.465785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:19.465847 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.466402 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:19.466407 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:19.468105 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:19.470102 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.470104 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.470105 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.470361 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.470580 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:19.470580 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:19.470581 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:19.470647 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:19.470922 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:19.470923 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:19.470973 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 02:14:19.471043 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 02:14:19.471151 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 02:14:19.473055 29015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8: Bootstrap starting.
I20260809 02:14:19.473896 29015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:19.474217 29015 log.cc:824] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:19.475379 29015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8: No bootstrap required, opened a new log
I20260809 02:14:19.476871 29015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } }
I20260809 02:14:19.477026 29015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 121bc67902f44f3085e4cae8fecb18f8, State: Initialized, Role: FOLLOWER
I20260809 02:14:19.477113 29015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } }
I20260809 02:14:19.477451 29015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:19.477486 28988 raft_consensus.cc:482] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 02:14:19.477559 28988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:19.478480 28988 raft_consensus.cc:504] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } }
I20260809 02:14:19.479166 29067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "121bc67902f44f3085e4cae8fecb18f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1ba25c4eb02149668cf1376bc524668e"
I20260809 02:14:19.479319 29067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:19.479796 29207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } } }
I20260809 02:14:19.479882 29207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:14:19.480245 29067 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 121bc67902f44f3085e4cae8fecb18f8 in term 1.
I20260809 02:14:19.480551 28961 leader_election.cc:302] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 121bc67902f44f3085e4cae8fecb18f8, 1ba25c4eb02149668cf1376bc524668e; no voters:
I20260809 02:14:19.480679 29207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:19.480820 29207 raft_consensus.cc:686] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [term 1 LEADER]: Becoming Leader. State: Replica: 121bc67902f44f3085e4cae8fecb18f8, State: Running, Role: LEADER
I20260809 02:14:19.480907 29207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } }
I20260809 02:14:19.481302 29207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 121bc67902f44f3085e4cae8fecb18f8. Latest consensus state: current_term: 1 leader_uuid: "121bc67902f44f3085e4cae8fecb18f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } } }
I20260809 02:14:19.481385 29207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:19.481564 29221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:19.482057 29221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:19.482708 28945 master_runner.cc:232] Existing masters have differing terms: 1 vs 0
I20260809 02:14:19.482782 28945 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:14:19.483078 29143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "121bc67902f44f3085e4cae8fecb18f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "66cbb472d379445eb8db3b94be7e5da3"
I20260809 02:14:19.483196 29143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:19.484050 29143 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 121bc67902f44f3085e4cae8fecb18f8 in term 1.
I20260809 02:14:19.484333 28988 leader_election.cc:288] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [CANDIDATE]: Term 1 election: Requested vote from peers 1ba25c4eb02149668cf1376bc524668e (127.5.9.189:33545), 66cbb472d379445eb8db3b94be7e5da3 (127.5.9.188:33345)
I20260809 02:14:19.498934 29143 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3 [term 1 FOLLOWER]: Refusing update from remote peer 121bc67902f44f3085e4cae8fecb18f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:19.499281 29067 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e [term 1 FOLLOWER]: Refusing update from remote peer 121bc67902f44f3085e4cae8fecb18f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:19.499886 29207 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:19.500083 29207 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:19.501413 29192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 121bc67902f44f3085e4cae8fecb18f8. Latest consensus state: current_term: 1 leader_uuid: "121bc67902f44f3085e4cae8fecb18f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } } }
I20260809 02:14:19.501514 29192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:14:19.502615 29175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 121bc67902f44f3085e4cae8fecb18f8. Latest consensus state: current_term: 1 leader_uuid: "121bc67902f44f3085e4cae8fecb18f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } } }
I20260809 02:14:19.502727 29175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:14:19.511701 29219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "121bc67902f44f3085e4cae8fecb18f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } } }
I20260809 02:14:19.511842 29219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:19.512442 29175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "121bc67902f44f3085e4cae8fecb18f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } } }
I20260809 02:14:19.512552 29175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:14:19.512852 29233 mvcc.cc:205] Tried to move back new op lower bound from 7316445837303541760 to 7316445837234388992. Current Snapshot: MvccSnapshot[applied={T|T < 7316445837303541760}]
I20260809 02:14:19.513198 29207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "121bc67902f44f3085e4cae8fecb18f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } } }
I20260809 02:14:19.513298 29207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:19.513696 29192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "121bc67902f44f3085e4cae8fecb18f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121bc67902f44f3085e4cae8fecb18f8" member_type: VOTER last_known_addr { host: "127.5.9.190" port: 37107 } } peers { permanent_uuid: "1ba25c4eb02149668cf1376bc524668e" member_type: VOTER last_known_addr { host: "127.5.9.189" port: 33545 } } peers { permanent_uuid: "66cbb472d379445eb8db3b94be7e5da3" member_type: VOTER last_known_addr { host: "127.5.9.188" port: 33345 } } }
I20260809 02:14:19.513794 29192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:14:19.514228 29097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.489972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59044 (local address 127.5.9.189:33545)
0809 02:14:19.490023 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.490025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.492081 (+ 2056us) server_negotiation.cc:407] Connection header received
0809 02:14:19.492178 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.492179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.492191 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.492220 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.494790 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.495456 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.500066 (+ 4610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.500379 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.500430 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.505440 (+ 5010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:19.505442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:19.505442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:19.505453 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:19.507327 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.510950 (+ 3623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.510952 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.510952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.511029 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.512004 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.512006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.512006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.512211 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:19.512322 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:19.514015 (+ 1693us) server_negotiation.cc:299] Negotiation successful
0809 02:14:19.514140 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 02:14:19.514508 29231 mvcc.cc:205] Tried to move back new op lower bound from 7316445837303541760 to 7316445837234388992. Current Snapshot: MvccSnapshot[applied={T|T < 7316445837303541760}]
I20260809 02:14:19.515488 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.490259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54342 (local address 127.5.9.190:37107)
0809 02:14:19.490298 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.490300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.491026 (+ 726us) server_negotiation.cc:407] Connection header received
0809 02:14:19.491129 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.491130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.491141 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.491172 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.492942 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.493707 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.497468 (+ 3761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.497777 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.497828 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.507785 (+ 9957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:19.507787 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:19.507787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:19.507797 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:19.509567 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.511287 (+ 1720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.511288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.511289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.511361 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.512700 (+ 1339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.512702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.512702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.512855 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:19.512931 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:19.515291 (+ 2360us) server_negotiation.cc:299] Negotiation successful
0809 02:14:19.515387 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:14:19.515496 29221 catalog_manager.cc:1347] Generated new cluster ID: c805dffecead43f7909460f1a926a5eb
I20260809 02:14:19.515632 29221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:19.534549 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:19.490757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55676 (local address 127.5.9.188:33345)
0809 02:14:19.490799 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.490800 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.496521 (+ 5721us) server_negotiation.cc:407] Connection header received
0809 02:14:19.496626 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.496627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.496639 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.496670 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.502880 (+ 6210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.503611 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.510051 (+ 6440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:19.510378 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:19.510433 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:19.531246 (+ 20813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:19.531249 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:19.531250 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:19.531260 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:19.533155 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.533638 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.533639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.533639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.533704 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:19.533917 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:19.533917 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:19.533918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:19.534056 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:19.534142 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:19.534365 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 02:14:19.534449 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:14:19.556947 29221 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:19.558892 29221 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:19.570484 29221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 121bc67902f44f3085e4cae8fecb18f8: Generated new TSK 0
I20260809 02:14:19.570838 29221 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:14:19.594835 29018 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:19.593853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54346 (local address 127.5.9.190:37107)
0809 02:14:19.593902 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.593904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.594079 (+ 175us) server_negotiation.cc:407] Connection header received
0809 02:14:19.594199 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.594200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.594212 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.594243 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.594751 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54346: BlockingRecv error: recv got EOF from 127.0.0.1:54346 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 02:14:19.595283 29097 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:19.594408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59048 (local address 127.5.9.189:33545)
0809 02:14:19.594509 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.594511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.594668 (+ 157us) server_negotiation.cc:407] Connection header received
0809 02:14:19.594767 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.594768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.594779 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.594808 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.595217 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59048: BlockingRecv error: recv got EOF from 127.0.0.1:59048 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 02:14:19.595629 29173 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:19.594464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55682 (local address 127.5.9.188:33345)
0809 02:14:19.594545 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.594547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.595077 (+ 530us) server_negotiation.cc:407] Connection header received
0809 02:14:19.595181 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.595182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.595195 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.595225 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.595566 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55682: BlockingRecv error: recv got EOF from 127.0.0.1:55682 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 02:14:19.597726 29097 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:19.597121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59054 (local address 127.5.9.189:33545)
0809 02:14:19.597165 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.597167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.597176 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:14:19.597249 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.597250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.597268 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.597332 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.597677 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59054: BlockingRecv error: recv got EOF from 127.0.0.1:59054 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 02:14:19.600350 29097 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:19.599832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59062 (local address 127.5.9.189:33545)
0809 02:14:19.599888 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.599890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.599898 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:14:19.599922 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.599923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.599933 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.599996 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.600300 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59062: BlockingRecv error: recv got EOF from 127.0.0.1:59062 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 02:14:19.604058 29097 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:19.603371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59076 (local address 127.5.9.189:33545)
0809 02:14:19.603412 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.603414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.603504 (+ 90us) server_negotiation.cc:407] Connection header received
0809 02:14:19.603542 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.603543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.603578 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.603643 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.604007 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59076: BlockingRecv error: recv got EOF from 127.0.0.1:59076 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 02:14:19.609809 29097 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:19.609097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59088 (local address 127.5.9.189:33545)
0809 02:14:19.609140 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.609142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.609201 (+ 59us) server_negotiation.cc:407] Connection header received
0809 02:14:19.609237 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.609237 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.609251 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.609327 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.609751 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59088: BlockingRecv error: recv got EOF from 127.0.0.1:59088 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 02:14:19.619666 29097 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:19.618943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59102 (local address 127.5.9.189:33545)
0809 02:14:19.619020 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.619022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.619105 (+ 83us) server_negotiation.cc:407] Connection header received
0809 02:14:19.619170 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.619171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.619185 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.619248 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.619602 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59102: BlockingRecv error: recv got EOF from 127.0.0.1:59102 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 02:14:19.640522 29097 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:19.636963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59104 (local address 127.5.9.189:33545)
0809 02:14:19.637081 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.637121 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.637334 (+ 213us) server_negotiation.cc:407] Connection header received
0809 02:14:19.637588 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.637590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.637608 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.637776 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.640450 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59104: BlockingRecv error: recv got EOF from 127.0.0.1:59104 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260809 02:14:19.677466 29097 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:19.676652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59116 (local address 127.5.9.189:33545)
0809 02:14:19.676753 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.676755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.676796 (+ 41us) server_negotiation.cc:407] Connection header received
0809 02:14:19.676875 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.676877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.676888 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.676938 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.677368 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59116: BlockingRecv error: recv got EOF from 127.0.0.1:59116 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 02:14:19.743531 29097 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:19.742837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59124 (local address 127.5.9.189:33545)
0809 02:14:19.742914 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.742920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.742991 (+ 71us) server_negotiation.cc:407] Connection header received
0809 02:14:19.743050 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.743051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.743062 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.743110 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.743444 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59124: BlockingRecv error: recv got EOF from 127.0.0.1:59124 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 02:14:19.873667 29097 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:19.872949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59138 (local address 127.5.9.189:33545)
0809 02:14:19.873021 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:19.873023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:19.873036 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:14:19.873085 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:19.873086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:19.873096 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:19.873164 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:19.873562 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59138: BlockingRecv error: recv got EOF from 127.0.0.1:59138 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 02:14:20.131904 29097 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:20.131122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59144 (local address 127.5.9.189:33545)
0809 02:14:20.131215 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:20.131218 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:20.131232 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:14:20.131291 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:20.131292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:20.131303 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:20.131375 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:20.131842 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59144: BlockingRecv error: recv got EOF from 127.0.0.1:59144 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:14:20.406375 29189 catalog_manager.cc:1259] Loaded cluster ID: c805dffecead43f7909460f1a926a5eb
I20260809 02:14:20.406472 29189 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3: loading cluster ID for follower catalog manager: success
I20260809 02:14:20.407260 29189 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3: acquiring CA information for follower catalog manager: success
I20260809 02:14:20.407532 29189 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 66cbb472d379445eb8db3b94be7e5da3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:14:20.424670 29203 catalog_manager.cc:1259] Loaded cluster ID: c805dffecead43f7909460f1a926a5eb
I20260809 02:14:20.424817 29203 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e: loading cluster ID for follower catalog manager: success
I20260809 02:14:20.425704 29203 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e: acquiring CA information for follower catalog manager: success
I20260809 02:14:20.426038 29203 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1ba25c4eb02149668cf1376bc524668e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:14:20.648576 29308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:20.645366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59160 (local address 127.5.9.189:33545)
0809 02:14:20.645513 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:20.645515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:20.645556 (+ 41us) server_negotiation.cc:407] Connection header received
0809 02:14:20.645723 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:20.645725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:20.645816 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:20.645943 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:20.646128 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:20.646700 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:20.647587 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:20.647840 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:20.647929 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:20.648146 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:20.648287 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:20.648388 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:20.648434 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 02:14:20.648461 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260809 02:14:20.653332 29314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:20.650722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55688 (local address 127.5.9.188:33345)
0809 02:14:20.650868 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:20.650870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:20.650893 (+ 23us) server_negotiation.cc:407] Connection header received
0809 02:14:20.650986 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:20.650987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:20.651025 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:20.651113 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:20.651247 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:20.651737 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:20.652479 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:20.652684 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:20.652773 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:20.652953 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:20.653070 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:20.653163 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:20.653202 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 02:14:20.653226 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":67,"threads_started":1}
I20260809 02:14:20.658797 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:20.655473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54362 (local address 127.5.9.190:37107)
0809 02:14:20.655747 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:20.655750 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:20.655758 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:14:20.655806 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:20.655807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:20.655864 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:20.655952 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:20.656141 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:20.656685 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:20.657659 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:20.657912 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:20.658003 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:20.658294 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:20.658419 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:20.658481 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:20.658655 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 02:14:20.658685 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":59,"threads_started":1}
I20260809 02:14:20.658851 29308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:20.655357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59162 (local address 127.5.9.189:33545)
0809 02:14:20.655416 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:20.655418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:20.655635 (+ 217us) server_negotiation.cc:407] Connection header received
0809 02:14:20.655705 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:20.655707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:20.655719 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:20.655823 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:20.655938 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:20.656681 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:20.657997 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:20.658300 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:20.658365 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:20.658564 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:20.658597 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:20.658627 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:20.658736 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 02:14:20.658773 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 02:14:20.658983 29314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:20.655556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55698 (local address 127.5.9.188:33345)
0809 02:14:20.655592 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:20.655594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:20.655699 (+ 105us) server_negotiation.cc:407] Connection header received
0809 02:14:20.655722 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:20.655723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:20.655734 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:20.655781 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:20.655957 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:20.656713 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:20.657976 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:20.658302 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:20.658385 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:20.658554 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:20.658590 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:20.658666 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:20.658876 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 02:14:20.658905 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 02:14:20.660884 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 28945
I20260809 02:14:20.668704 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 29023
I20260809 02:14:20.675149 5158 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktks196/build/release/bin/kudu with pid 29100
2026-08-09T02:14:20Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3026 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6004 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (283282 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST